Inducing Bilingual Lexica From Non-Parallel Data With Earth Mover’s Distance Regularization

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3188–3198, Osaka, Japan, December 11-17 2016. Inducing Bilingual Lexica From Non-Parallel Data With Earth Mover’s Distance Regularization Meng Zhang†‡ Yang Liu†‡ Huanbo Luan† Yiqun Liu† Maosong Sun†‡ †State Key Laboratory of Intelligent Technology and Systems Tsinghua National Laboratory for Information Science and Technology Department of Computer Science and Technology, Tsinghua University, Beijing, China ‡Jiangsu Collaborative Innovation Center for Language Competence, Jiangsu, China zmlarry@foxmail.com, liuyang2011@tsinghua.edu.cn luanhuanbo@gmail.com, {yiqunliu, sms}@tsinghua.edu.cn Abstract Being able to induce word translations from non-parallel data is often a prerequisite for cross- lingual processing in resource-scarce languages and domains. Previous endeavors typically sim- plify this task by imposing the one-to-one translation assumption, which is too strong to hold for natural languages. We remove this constraint by introducing the Earth Mover’s Distance into the training of bilingual word embeddings. In this way, we take advantage of its capability to handle multiple alternative word translations in a natural form of regularization. Our approach shows significant and consistent improvements across four language pairs. We also demonstrate that our approach is particularly preferable in resource-scarce settings as it only requires a minimal seed lexicon. 1 Introduction Bilingual lexica provide word-level semantic equivalence information across languages, and prove to be valuable for a range of cross-lingual natural language processing tasks (Och and Ney, 2003; Levow et al., 2005; T¨ackstr¨om et al., 2013, inter alia). As building bilingual lexica from parallel corpora has been solved by word alignment (Och and Ney, 2003), researchers have turned their attention to non-parallel corpora. Accompanied by a small seed lexicon, non-parallel corpora are usually the only resources available in resource-scarce languages and domains, making the task of bilingual lexicon induction both important and challenging. A variety of statistical methods have been proposed to induce bilingual lexica from non-parallel data (Rapp, 1999; Koehn and Knight, 2002; Fung and Cheung, 2004; Gaussier et al., 2004; Haghighi et al., 2008; Ravi and Knight, 2011; Vuli´c et al., 2011; Vuli´c and Moens, 2013a; Vuli´c and Moens, 2013b; Dong et al., 2015). With the surge of word embeddings trained by neural networks, recent approaches that learn bilingual word representations from non-parallel data for bilingual lexicon induction have also shown promise (Mikolov et al., 2013b; Vuli´c and Moens, 2015). However, none of the existing methods explicitly considers multiple alternative translation, i.e., the phenomenon that one source language word may have multiple possible translations in the target lan- guage. For example, the (romanized) Chinese word “qiche” can be translated to “car” or “automobile” in English, while the English word “car” can mean “qiche” or “chexiang” (railway carriage) in Chinese. Al- though prevalent among natural languages (Resnik and Yarowsky, 1999), multiple alternative translation is basically ignored by prior bilingual lexicon inducers; instead, they typically impose the one-to-one translation assumption (Vuli´c and Moens, 2013b) for simplicity. This represents a major drawback of existing bilingual lexicon induction approaches. There has been one study that shows potential for tackling this issue. It introduces the Earth Mover’s …

Content truncated to 500 words