MAAM: A Morphology-Aware Alignment Model for Unsupervised Bilingual Lexicon Induction

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3190–3196 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3190 MAAM: A Morphology-Aware Alignment Model for Unsupervised Bilingual Lexicon Induction Pengcheng Yang1,2, Fuli Luo2, Peng Chen2, Tianyu Liu2, Xu Sun1,2 1Deep Learning Lab, Beijing Institute of Big Data Research, Peking University 2MOE Key Lab of Computational Linguistics, School of EECS, Peking University {yang pc, luofuli, chen.peng, tianyu0421, xusun}@pku.edu.cn Abstract The task of unsupervised bilingual lexicon in- duction (UBLI) aims to induce word transla- tions from monolingual corpora in two lan- guages. Previous work has shown that mor- phological variation is an intractable challenge for the UBLI task, where the induced transla- tion in failure case is usually morphologically related to the correct translation. To tackle this challenge, we propose a morphology-aware alignment model for the UBLI task. The pro- posed model aims to alleviate the adverse ef- fect of morphological variation by introduc- ing grammatical information learned by the pre-trained denoising language model. Results show that our approach can substantially out- perform several state-of-the-art unsupervised systems, and even achieves competitive per- formance compared to supervised methods. 1 Introduction The task of unsupervised bilingual lexicon induc- tion aims at identifying translational equivalents across two languages (Kementchedjhieva et al., 2018). It can be applied in plenty of real-scenarios, such as machine translation (Artetxe et al., 2018b), transfer learning (Zhou et al., 2016), and so on. Based on the observation that embedding spaces of different languages exhibit similar structures, a prominent approach is to align monolingual em- bedding spaces of two languages with a simple lin- ear mapping (Zhang et al., 2017a; Lample et al., 2018). However, previous work (Artetxe et al., 2018a; Søgaard et al., 2018) has shown that mor- phological variation is an intractable challenge for the UBLI task. The induced translations in failure cases are usually morphologically related words. Due to similar semantics, these words can easily confuse the system to make the incorrect align- ment. Table 1 presents three randomly selected failure examples of MUSE (Lample et al., 2018) Source word Top-3 of retrieved nearest neighbors mangez eats eat buttered suspendit suspending suspend suspended diffusant broadcasts broadcast broadcasting Table 1: Three randomly selected failure examples of MUSE on FR-EN language pair. Red words are correct translations, which are all not the nearest translations. on the FR-EN language pair, showing that all failures can be attributed to morphological vari- ation. For instance, for the French source word “mangez”, MUSE translates it to morphologically related word “eats”, instead of the correct English translation “eat”. However, we find that additional grammatical information can help alleviate the adverse effect of morphological variation. In detail, since lexi- con induction (word alignment) can be regarded as word-to-word translation, the fluency of the translated sentence can reflect the quality of word alignment. If the model can retrieve the correct translation for each word in a source sentence, the translated sentence is more likely to be flu- ent and …

Content truncated to 500 words