Don’t Forget the Long Tail! A Comprehensive Analysis of Morphological Generalization in Bilingual Lexicon Induction

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 974–983, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 974 Don’t Forget the Long Tail! A Comprehensive Analysis of Morphological Generalization in Bilingual Lexicon Induction Paula Czarnowska1 Sebastian Ruder2∗ Edouard Grave3 Ryan Cotterell1 Ann Copestake1 1University of Cambridge 2National University of Ireland 3Facebook AI Research pjc211@cam.ac.uk sebastian@ruder.io egrave@fb.com rdc42@cam.ac.uk aac10@cam.ac.uk Abstract Human translators routinely have to translate rare inflections of words—due to the Zipfian distribution of words in a language. When translating from Spanish, a good translator would have no problem identifying the proper translation of a statistically rare inflection such as hablar´amos. Note the lexeme itself, hablar, is relatively common. In this work, we investi- gate whether state-of-the-art bilingual lexicon inducers are capable of learning this kind of generalization. We introduce 40 morpholog- ically complete dictionaries in 10 languages1 and evaluate three of the state-of-the-art mod- els on the task of translation of less frequent morphological forms. We demonstrate that the performance of state-of-the-art models drops considerably when evaluated on infrequent morphological inflections and then show that adding a simple morphological constraint at training time improves the performance, proving that the bilingual lexicon inducers can benefit from better encoding of morphology. 1 Introduction Human translators exhibit remarkable general- ization capabilities and are able to translate even rare inflections they may have never seen before. Indeed, this skill is necessary for translation since language follows a Zipfian distribution (Zipf, 1949): a large number of the tokens in a translated text will come from rare types, including rare inflections of common lexemes. For instance, a Spanish translator will most certainly know the verb hablar “to speak”, but they will only have seen the less frequent, first-person plural future form hablar´amos a few times. Nevertheless, they would have no problem translating the latter. In this paper we ask whether current methods for bilingual lexi- con induction (BLI) generalize morphologically as 1The dictionaries are available at https://github. com/pczarnowska/morph_dictionaries. ⋆Sebastian is now affiliated with DeepMind. –10k 10–50k 50–100k 100–200k 200–300k 300–400k 400–500k 500–600k Remaining OOVs 1163 970 2506 - 1512 - 1470 - 745 - 494 - 416 - 338 - 1126 - 4917 - Dictionary Ours MUSE In vocab +OOVs 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Accuracy 9770 970 21087 970 French–Spanish Figure 1: The relation between the BLI performance and the frequency of source words in the test dictio- nary. The graph presents results for the model of Ruder et al. (2018) evaluated on both the MUSE dictionary (Conneau et al., 2018) and our morphologically com- plete dictionary, which contains many rare morpholog- ical variants of words. The numbers above the bars cor- respond to the number of translated source words (a hyphen represents an empty dictionary). humans do. Generalization to rare and novel words is arguably the main point of BLI as a task—most frequent translation pairs are already contained in …

Content truncated to 500 words