Probabilistic Models of Cross-Lingual Semantic Similarity in Context Based on Latent Cross-Lingual Concepts Induced from Comparable Data

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 349–362, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Probabilistic Models of Cross-Lingual Semantic Similarity in Context Based on Latent Cross-Lingual Concepts Induced from Comparable Data Ivan Vuli´c and Marie-Francine Moens Department of Computer Science KU Leuven, Belgium {ivan.vulic|marie-francine.moens}@cs.kuleuven.be Abstract We propose the first probabilistic approach to modeling cross-lingual semantic sim- ilarity (CLSS) in context which requires only comparable data. The approach re- lies on an idea of projecting words and sets of words into a shared latent semantic space spanned by language-pair indepen- dent latent semantic concepts (e.g., cross- lingual topics obtained by a multilingual topic model). These latent cross-lingual concepts are induced from a comparable corpus without any additional lexical re- sources. Word meaning is represented as a probability distribution over the latent concepts, and a change in meaning is rep- resented as a change in the distribution over these latent concepts. We present new models that modulate the isolated out-of- context word representations with contex- tual knowledge. Results on the task of suggesting word translations in context for 3 language pairs reveal the utility of the proposed contextualized models of cross- lingual semantic similarity. 1 Introduction Cross-lingual semantic similarity (CLSS) is a met- ric that measures to which extent words (or more generally, text units) describe similar semantic concepts and convey similar meanings across lan- guages. Models of cross-lingual similarity are typ- ically used to automatically induce bilingual lexi- cons and have found numerous applications in in- formation retrieval (IR), statistical machine trans- lation (SMT) and other natural language process- ing (NLP) tasks. Within the IR framework, the output of the CLSS models is a key resource in the models of dictionary-based cross-lingual in- formation retrieval (Ballesteros and Croft, 1997; Lavrenko et al., 2002; Levow et al., 2005; Wang and Oard, 2006) or may be utilized in query ex- pansion in cross-lingual IR models (Adriani and van Rijsbergen, 1999; Vuli´c et al., 2013). These CLSS models may also be utilized as an addi- tional source of knowledge in SMT systems (Och and Ney, 2003; Wu et al., 2008). Additionally, the models are a crucial component in the cross- lingual tasks involving a sort of cross-lingual knowledge transfer, where the knowledge about utterances in one language may be transferred to another. The utility of the transfer or annotation projection by means of bilingual lexicons obtained from the CLSS models has already been proven in various tasks such as semantic role labeling (Pad´o and Lapata, 2009; van der Plas et al., 2011), parsing (Zhao et al., 2009; Durrett et al., 2012; T¨ackstr¨om et al., 2013b), POS tagging (Yarowsky and Ngai, 2001; Das and Petrov, 2011; T¨ackstr¨om et al., 2013a; Ganchev and Das, 2013), verb clas- sification (Merlo et al., 2002), inducing selectional preferences (Peirsman and Pad´o, 2010), named entity recognition (Kim et al., 2012), named en- tity segmentation (Ganchev and Das, 2013), etc. The models of cross-lingual semantic similar- ity from parallel corpora …

Content truncated to 500 words