Classification-Based Self-Learning for Weakly Supervised Bilingual Lexicon Induction

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6915–6922 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 6915 Classification-Based Self-Learning for Weakly Supervised Bilingual Lexicon Induction Mladen Karan♠∗ Ivan Vuli´c♦∗ Anna Korhonen♦ Goran Glavaˇs♣ ♠TakeLab, Faculty of Electrical Engineering and Computing, University of Zagreb ♦ Language Technology Lab, TAL, University of Cambridge ♣ Data and Web Science Group, University of Mannheim mladen.karan@fer.hr {iv250,alk23}@cam.ac.uk goran@informatik.uni-mannheim.de Abstract Effective projection-based cross-lingual word embedding (CLWE) induction critically relies on the iterative self-learning procedure. gradually expands the initial small seed dictio-It nary to learn improved cross-lingual mappings. In this work, we present CLASSYMAP, a classification-based approach to self-learning, yielding a more robust and a more effective in- duction of projection-based CLWEs. Unlike prior self-learning methods, our approach al- lows for integration of diverse features into the iterative process. We show the benefits of CLASSYMAP for bilingual lexicon induction: we report consistent improvements in a weakly supervised setup (500 seed translation pairs) on a benchmark with 28 language pairs. 1 Introduction and Motivation Cross-lingual word embeddings (CLWEs), that is, representations of words in a shared cross- lingual vector space, enable multilingual model- ing of meaning and facilitate cross-lingual transfer for downstream NLP tasks (Ruder et al., 2019). One of their primary use cases is bilingual lexi- con induction (BLI), that is, learning translation correspondences across languages which benefit the development of core language technology also for resource-poor languages and domains (Adams et al., 2017; Smith et al., 2017; Heyman et al., 2018; Hangya et al., 2018; Vuli´c et al., 2019). Earlier work focused on joint CLWE induction from bilingual corpora, relying on word- (Kle- mentiev et al., 2012; Gouws and Søgaard, 2015), sentence- (Zou et al., 2013; Hermann and Blunsom, 2014; Coulmance et al., 2015; Levy et al., 2017), or document-level supervision (Søgaard et al., 2015; Vuli´c and Moens, 2016). However, recent focus is predominantly on post-hoc alignment of indepen- dently trained monolingual word embeddings: the ∗Equal contribution. so-called projection-based or mapping approaches (Mikolov et al., 2013; Conneau et al., 2018; Joulin et al., 2018; Artetxe et al., 2018b; Patra et al., 2019). Such methods are particularly suitable for weakly supervised learning setups: they support CLWE induction with only as much as few thousand word translation pairs as the bilingual supervision.1 One critical component of weakly supervised projection-based CLWEs is a self-learning proce- dure that iteratively refines the initial seed dictio- nary to learn projections of increasingly higher quality. This process leads to substantial improve- ments of the initially mapped space, especially with smaller seed dictionaries (Artetxe et al., 2017; Vuli´c et al., 2019). However, current self-learning procedures are still rather basic, typically rely- ing only on direct extraction of (mutual) nearest neighbors from the current shared space (Conneau et al., 2018; Artetxe et al., 2018b; Glavaˇs et al., 2019). In this work, we propose a more sophis- ticated self-learning procedure for weakly super- vised projection-based CLWE methods, and show its benefits for a wide range of language pairs. …

Content truncated to 500 words