Bilingual Lexicon Induction by Learning to Combine Word-Level and Character-Level Representations

Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 1085–1095, Valencia, Spain, April 3-7, 2017. c⃝2017 Association for Computational Linguistics Bilingual Lexicon Induction by Learning to Combine Word-Level and Character-Level Representations Geert Heyman1, Ivan Vuli´c2, and Marie-Francine Moens1 1LIIR, Department of Computer Science, KU Leuven 2Language Technology Lab, DTAL, University of Cambridge {geert.heyman,sien.moens}@cs.kuleuven.be iv250@cam.ac.uk Abstract We study the problem of bilingual lexicon induction (BLI) in a setting where some translation resources are available, but un- known translations are sought for certain, possibly domain-specific terminology. We frame BLI as a classification problem for which we design a neural network based classification architecture composed of re- current long short-term memory and deep feed forward networks. The results show that word- and character-level representa- tions each improve state-of-the-art results for BLI, and the best results are obtained by exploiting the synergy between these word- and character-level representations in the classification model. 1 Introduction Bilingual lexicon induction (BLI) is the task of find- ing words that share a common meaning across dif- ferent languages. Automatically induced bilingual lexicons support a variety of tasks in information retrieval and natural language processing, includ- ing cross-lingual information retrieval (Lavrenko et al., 2002; Levow et al., 2005; Vuli´c and Moens, 2015; Mitra et al., 2016), statistical machine trans- lation (Och and Ney, 2003; Zou et al., 2013), or cross-lingual entity linking (Tsai and Roth, 2016). In addition, they serve as a natural bridge for cross-lingual annotation and model transfer from resource-rich to resource-impoverished languages, finding their application in downstream tasks such as cross-lingual POS tagging (Yarowsky and Ngai, 2001; T¨ackstr¨om et al., 2013; Zhang et al., 2016), dependency parsing (Zhao et al., 2009; Durrett et al., 2012; Upadhyay et al., 2016), semantic role labeling (Pad´o and Lapata, 2009; van der Plas et al., 2011), to name only a few. Current state-of-the-art BLI results are obtained by cross-lingual word embeddings (Mikolov et al., 2013b; Faruqui and Dyer, 2014; Gouws et al., 2015; Vuli´c and Moens, 2016; Duong et al., 2016, inter alia). They significantly outperform tradi- tional count-based baselines (Gaussier et al., 2004; Tamura et al., 2012). Although cross-lingual word embedding models differ on the basis of a bilingual signal from parallel, comparable or monolingual data used in training (e.g., word, sentence, docu- ment alignments, translation pairs from a seed lexi- con),1 they all induce word translations in the same manner. (1) They learn a shared bilingual semantic space in which all source language and target lan- guage words are represented as dense real-valued vectors. The shared space enables words from both languages to be represented in a uniform language- independent manner such that similar words (re- gardless of the actual language) have similar rep- resentations. (2) Cross-lingual semantic similar- ity between words w and v is then computed as SF(⃗w,⃗v), where ⃗w and ⃗v are word representations in the shared space, and SF denotes a similarity function operating in the space (cosine similarity is typically used). A …

Content truncated to 500 words