A Relaxed Matching Procedure for Unsupervised BLI

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3036–3041 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 3036 A Relaxed Matching Procedure for Unsupervised BLI Xu Zhao1, Zihao Wang1, Hao Wu2, Zhang Yong1 † 1BNRist, Department of Computer Science and Technology, RIIT, Institute of Internet Industry, Tsinghua University, Beijing, China 2Department of Mathematical Sciences, Tsinghua University, Beijing, China {zhaoxu18, wzh17}@mails.tsinghua.edu.cn {hwu, zhangyong05}@tsinghua.edu.cn Abstract Recently unsupervised Bilingual Lexicon In- duction(BLI) without any parallel corpus has attracted much research interest. One of the crucial parts in methods for the BLI task is the matching procedure. Previous works impose a too strong constraint on the matching and lead to many counterintuitive translation pairings. Thus, We propose a relaxed matching proce- dure to find a more precise matching between two languages. We also find that aligning source and target language embedding space bidirectionally will bring significant improve- ment. We follow the previous iterative frame- work to conduct experiments. Results on stan- dard benchmark demonstrate the effectiveness of our proposed method, which substantially outperforms previous unsupervised methods. 1 Introduction Pretrained word embeddings (Mikolov et al., 2013b) are the basis of many other natural lan- guage processing and machine learning systems. Word embeddings of a specific language contain rich syntax and semantic information. Mikolov et al. (2013a) stated that the continuous embedding spaces exhibit similar structures across different languages, and we can exploit the similarity by a linear transformation from source embedding space to target embedding space. This similarity derives the Bilingual Lexicon Induction(BLI) task. The goal of bilingual lexicon induction is to align two languages’ embedding space and generates word translation lexicon automatically. This fundamen- tal problem in natural language processing benefits much other research such as sentence translation (Rapp, 1995; Fung, 1995), unsupervised machine translation (Lample et al., 2017), cross-lingual in- formation retrieval (Lavrenko et al., 2002). Recent endeavors (Lample et al., 2018; Alvarez- Melis and Jaakkola, 2018; Grave et al., 2019; †Yong Zhang is the corresponding author. Artetxe et al., 2017) have proven that unsupervised BLI’s performance is even on par with the super- vised methods. A crucial part of these approaches is the matching procedure, i.e., how to generate the translation plan. Alvarez-Melis and Jaakkola (2018) used Gromov-Wasserstein distance to ap- proximate the matching between languages. Grave et al. (2019) regarded it as a classic optimal trans- port problem and used the sinkhorn algorithm (Cu- turi, 2013) to compute the translation plan. In this work, we follow the previous iterative framework but use a different matching procedure. Previous iterative algorithms required to compute an approximate 1 to 1 matching every step. This 1 to 1 constraint brings out many redundant match- ings. Thus in order to avoid this problem, we relax the constraint and control the relaxation degree by adding two KL divergence regularization terms to the original loss function. This relaxation derives a more precise matching and significantly improves performance. Then we propose a bidirectional opti- mization framework to optimize the mapping from source …

Content truncated to 500 words