A Graph-based Coarse-to-fine Method for Unsupervised Bilingual Lexicon Induction

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3476–3485 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 3476 A Graph-based Coarse-to-fine Method for Unsupervised Bilingual Lexicon Induction Shuo Ren†‡∗, Shujie Liu§, Ming Zhou§, Shuai Ma†‡ †SKLSDE Lab, Beihang University, Beijing, China ‡Beijing Advanced Innovation Center for Big Data and Brain Computing, China §Microsoft Research Asia, Beijing, China †{shuoren,mashuai}@buaa.edu.cn §{shujliu,mingzhou}@microsoft.com Abstract Unsupervised bilingual lexicon induction is the task of inducing word translations from monolingual corpora of two languages. Re- cent methods are mostly based on unsuper- vised cross-lingual word embeddings, the key to which is to find initial solutions of word translations, followed by the learning and re- finement of mappings between the embedding spaces of two languages. However, previous methods find initial solutions just based on word-level information, which may be (1) lim- ited and inaccurate, and (2) prone to contain some noise introduced by the insufficiently pre-trained embeddings of some words. To deal with those issues, in this paper, we pro- pose a novel graph-based paradigm to induce bilingual lexicons in a coarse-to-fine way. We first build a graph for each language with its vertices representing different words. Then we extract word cliques from the graphs and map the cliques of two languages. Based on that, we induce the initial word translation solution with the central words of the aligned cliques. This coarse-to-fine approach not only lever- ages clique-level information, which is richer and more accurate, but also effectively reduces the bad effect of the noise in the pre-trained embeddings. Finally, we take the initial solu- tion as the seed to learn cross-lingual embed- dings, from which we induce bilingual lexi- cons. Experiments show that our approach im- proves the performance of bilingual lexicon in- duction compared with previous methods. 1 Introduction Bilingual lexicon induction (BLI) is an important task of machine translation and becomes an essen- tial part of recent unsupervised machine translation approaches (Lample et al., 2018; Artetxe et al., 2018c; Marie and Fujita, 2018; Ren et al., 2019; Artetxe et al., 2019). Previous methods for BLI are ∗Contribution during internship at MSRA. mostly based on unsupervised cross-lingual word embeddings (Zhang et al., 2017; Artetxe et al., 2017; Conneau et al., 2017; Artetxe et al., 2018b; Xu et al., 2018; Hoshen and Wolf, 2018; Alvarez- Melis and Jaakkola, 2018), the goal of which is to find a mapping function, typically a linear transfor- mation (Mikolov et al., 2013), to map the source embeddings into the target embedding spaces. To do this, they first build a seed dictionary (known as the initial solution) with different methods and then learn the optimal mapping function that fits the seed dictionary. Based on the mapping function, a new dictionary of higher quality is inferred from the cross-lingual word embeddings by finding near- est neighbors in the target embedding space. With the new dictionary, the mapping function is further refined to fit it. The inference of the dictionary and the refinement of the mapping function …

Content truncated to 500 words