Accurate Cross-lingual Projection between Count-based Word Vectors by Exploiting Translatable Context Pairs

Proceedings of the 19th Conference on Computational Language Learning, pages 300–304, Beijing, China, July 30-31, 2015. c⃝2015 Association for Computational Linguistics Accurate Cross-lingual Projection between Count-based Word Vectors by Exploiting Translatable Context Pairs Shonosuke Ishiwatari♣, Nobuhiro Kaji♦♥, Naoki Yoshinaga♦♥, ♣ Graduate School of Information Science and Technology, the University of Tokyo ♦ Institute of Industrial Science, the University of Tokyo ♥ National Institute of Information and Communications Technology, Japan {ishiwatari, kaji, ynaga}@tkl.iis.u-tokyo.ac.jp Masashi Toyoda♦, Masaru Kitsuregawa♦♠ ♦ Institute of Industrial Science, the University of Tokyo ♠ National Institute of Informatics, Japan {toyoda, kitsure}@tkl.iis.u-tokyo.ac.jp Abstract We propose a method that learns a cross- lingual projection of word representations from one language into another. Our method utilizes translatable context pairs as bonus terms of the objective function. In the experiments, our method outper- formed existing methods in three language pairs, (English, Spanish), (Japanese, Chi- nese) and (English, Japanese), without us- ing any additional supervisions. 1 Introduction Vector-based representations of word meanings, hereafter word vectors, have been widely used in a variety of NLP applications including synonym detection (Baroni et al., 2014), paraphrase detec- tion (Erk and Pad´o, 2008), and dialogue analy- sis (Kalchbrenner and Blunsom, 2013). The ba- sic idea behind those representation methods is the distributional hypothesis (Harris, 1954; Firth, 1957) that similar words are likely to co-occur with similar context words. A problem with the word vectors is that they are not meant for capturing the similarity between words in different languages, i.e., translation pairs such as “gato” and “cat.” The meaning represen- tations of such word pairs are usually dissimilar, because the vast majority of the context words are from the same language as the target words (e.g., Spanish for “gato” and English for “cat”). This prevents using word vectors in multi-lingual appli- cations such as cross-lingual information retrieval and machine translation. Several approaches have been made so far to address this problem (Fung, 1998; Klementiev et al., 2012; Mikolov et al., 2013b). In particular, Mikolov et al. (2013b) recently explored learning a linear transformation between word vectors of different languages from a small amount of train- ing data, i.e., a set of bilingual word pairs. This study explores incorporating prior knowl- edge about the correspondence between dimen- sions of word vectors to learn more accurate trans- formation, when using count-based word vectors (Baroni et al., 2014). Since the dimensions of count-based word vectors are explicitly associated with context words, we can partially be aware of the cross-lingual correspondence between the di- mensions of word vectors by diverting the training data. Also, word surface forms present noisy yet useful clues on the correspondence when targeting the language pairs that have exchanged their vo- cabulary (e.g., “cocktail” in English and “c´octel” in Spanish). Although apparently useful, how to exploit such knowledge within the learning frame- work has not been addressed so far. We evaluated the proposed method in three lan- guage pairs. Compared with baselines including a method that uses vectors learned by neural net- works, our method gave better …

Content truncated to 500 words