A Study on Bootstrapping Bilingual Vector Spaces from Non-Parallel Data (and Nothing Else)

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1613–1624, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics A Study on Bootstrapping Bilingual Vector Spaces from Non-Parallel Data (and Nothing Else) Ivan Vuli´c and Marie-Francine Moens Department of Computer Science KU Leuven Celestijnenlaan 200A Leuven, Belgium {ivan.vulic,marie-francine.moens}@cs.kuleuven.be Abstract We present a new language pair agnostic ap- proach to inducing bilingual vector spaces from non-parallel data without any other re- source in a bootstrapping fashion. The pa- per systematically introduces and describes all key elements of the bootstrapping procedure: (1) starting point or seed lexicon, (2) the confi- dence estimation and selection of new dimen- sions of the space, and (3) convergence. We test the quality of the induced bilingual vec- tor spaces, and analyze the influence of the different components of the bootstrapping ap- proach in the task of bilingual lexicon extrac- tion (BLE) for two language pairs. Results re- veal that, contrary to conclusions from prior work, the seeding of the bootstrapping pro- cess has a heavy impact on the quality of the learned lexicons. We also show that our ap- proach outperforms the best performing fully corpus-based BLE methods on these test sets. 1 Introduction Bilingual lexicons serve as an indispensable source of knowledge for various cross-lingual tasks such as cross-lingual information retrieval (Lavrenko et al., 2002; Levow et al., 2005) or statistical machine translation (Och and Ney, 2003). Additionally, they are a crucial component in cross-lingual knowledge transfer, where the knowledge about utterances in one language may be transferred to another. The utility of the transfer or annotation projection by means of bilingual lexicons has already been proven in various tasks such as semantic role labeling (Pad´o and Lapata, 2009; van der Plas et al., 2011), parsing (Zhao et al., 2009; Durrett et al., 2012; T¨ackstr¨om et al., 2013b), POS tagging (Yarowsky and Ngai, 2001; Das and Petrov, 2011; T¨ackstr¨om et al., 2013a), etc. Techniques for automatic bilingual lexicon ex- traction (BLE) from parallel corpora on the basis of word alignment models are well established (Och and Ney, 2003). However, due to a relative scarce- ness of parallel data for many language pairs and domains, alternative approaches that rely on compa- rable corpora have also gained much interest (e.g., Fung and Yee (1998); Rapp (1999)). The models that rely on non-parallel data typ- ically represent each word by a high-dimensional vector in a feature vector space, where the dimen- sions of the vector are its context features. The con- text features are typically words co-occurring with the word in a predefined context.1 The similar- ity of two words, wS 1 given in the source language LS with vocabulary V S and wT 2 in the target lan- guage LT with vocabulary V T is then computed as sim(wS 1 , wT 2 ) = SF(cv(wS 1 ), cv(wT 2 )). cv(wS 1 ) = [scS 1 (c1), . . . , scS 1 (cN)] is a context vector for wS …

Content truncated to 500 words