Bilingual Lexicon Induction through Unsupervised Machine Translation

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5002–5007 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5002 Bilingual Lexicon Induction through Unsupervised Machine Translation Mikel Artetxe, Gorka Labaka, Eneko Agirre IXA NLP Group University of the Basque Country (UPV/EHU) {mikel.artetxe, gorka.labaka, e.agirre}@ehu.eus Abstract A recent research line has obtained strong re- sults on bilingual lexicon induction by align- ing independently trained word embeddings in two languages and using the resulting cross- lingual embeddings to induce word translation pairs through nearest neighbor or related re- trieval methods. In this paper, we propose an alternative approach to this problem that builds on the recent work on unsupervised machine translation. This way, instead of directly in- ducing a bilingual lexicon from cross-lingual embeddings, we use them to build a phrase- table, combine it with a language model, and use the resulting machine translation system to generate a synthetic parallel corpus, from which we extract the bilingual lexicon us- ing statistical word alignment techniques. As such, our method can work with any word em- bedding and cross-lingual mapping technique, and it does not require any additional resource besides the monolingual corpus used to train the embeddings. When evaluated on the exact same cross-lingual embeddings, our proposed method obtains an average improvement of 6 accuracy points over nearest neighbor and 4 points over CSLS retrieval, establishing a new state-of-the-art in the standard MUSE dataset. 1 Introduction Cross-lingual word embedding mappings have at- tracted a lot of attention in recent times. These methods work by independently training word embeddings in different languages, and mapping them to a shared space through linear transforma- tions. While early methods required a training dic- tionary to find the initial alignment (Mikolov et al., 2013), fully unsupervised methods have managed to obtain comparable results based on either ad- versarial training (Conneau et al., 2018) or self- learning (Artetxe et al., 2018b). A prominent application of these methods is Bilingual Lexicon Induction (BLI), that is, using the resulting cross-lingual embeddings to build a bilingual dictionary. For that purpose, one would typically induce the translation of each source word by taking its corresponding nearest neigh- bor in the target language. However, it has been argued that this basic approach suffers from the hubness problem1, which has motivated alterna- tive retrieval methods like inverted nearest neigh- bor2 (Dinu et al., 2015), inverted softmax (Smith et al., 2017), and Cross-domain Similarity Local Scaling (CSLS) (Conneau et al., 2018). In this paper, we go one step further and, rather than directly inducing the bilingual dictio- nary from the cross-lingual word embeddings, we use them to build an unsupervised machine trans- lation system, and extract a bilingual dictionary from a synthetic parallel corpus generated with it. This allows us to take advantage of a strong language model and naturally extract translation equivalences through statistical word alignment. At the same time, our method can be used as a drop-in replacement of traditional retrieval tech- niques, as it …

Content truncated to 500 words