Learning to Represent Bilingual Dictionaries

Proceedings of the 23rd Conference on Computational Natural Language Learning, pages 152–162 Hong Kong, China, November 3-4, 2019. c⃝2019 Association for Computational Linguistics 152 Learning to Represent Bilingual Dictionaries Muhao Chen1∗, Yingtao Tian2∗, Haochen Chen2, Kai-Wei Chang1, Steven Skiena2 & Carlo Zaniolo1 1University of California, Los Angeles, CA, USA 2The State University of New York, Stony Brook, NY, USA muhaochen@ucla.edu; {kwchang, zaniolo}@cs.ucla.edu; {yittian, haocchen, skiena}@cs.stonybrook.edu Abstract Bilingual word embeddings have been widely used to capture the correspondence of lexi- cal semantics in different human languages. However, the cross-lingual correspondence between sentences and words is less studied, despite that this correspondence can signifi- cantly benefit many applications such as cross- lingual semantic search and textual inference. To bridge this gap, we propose a neural em- bedding model that leverages bilingual dictio- naries1. The proposed model is trained to map the lexical definitions to the cross-lingual tar- get words, for which we explore with differ- ent sentence encoding techniques. To enhance the learning process on limited resources, our model adopts several critical learning strate- gies, including multi-task learning on differ- ent bridges of languages, and joint learning of the dictionary model with a bilingual word em- bedding model. We conduct experiments on two new tasks. In the cross-lingual reverse dictionary retrieval task, we demonstrate that our model is capable of comprehending bilin- gual concepts based on descriptions, and the proposed learning strategies are effective. In the bilingual paraphrase identification task, we show that our model effectively associates sen- tences in different languages via a shared em- bedding space, and outperforms existing ap- proaches in identifying bilingual paraphrases. 1 Introduction Cross-lingual semantic representation learning has attracted significant attention recently. Various ap- proaches have been proposed to align words of different languages in a shared embedding space (Ruder et al., 2017). By offering task-invariant se- ∗ Both authors contributed equally to this work. 1We refer the term dictionary to its common meaning, i.e. lexical definitions of words. Note that this is different from some papers on bilingual settings that refer dictionaries to seed lexicons for one-to-one word mappings. mantic transfers, these approaches critically sup- port many cross-lingual NLP tasks including neu- ral machine translations (NMT) (Devlin et al., 2014), bilingual document classification (Zhou et al., 2016), knowledge alignment (Chen et al., 2018b) and entity linking (Upadhyay et al., 2018). While many existing approaches have been pro- posed to associate lexical semantics between lan- guages (Chandar et al., 2014; Gouws et al., 2015; Luong et al., 2015a), modeling the correspon- dence between lexical and sentential semantics across different languages is still an unresolved challenge. We argue that learning to represent such cross-lingual and multi-granular correspon- dence is well desired and natural for multiple rea- sons. One reason is that, learning word-to-word correspondence has a natural limitation, consider- ing that many words do not have direct transla- tions in another language. For example, schaden- freude in German, which means a feeling of joy that comes from knowing the troubles of other people, has no proper English …

Content truncated to 500 words