Generalizing Procrustes Analysis for Better Bilingual Dictionary Induction

Proceedings of the 22nd Conference on Computational Natural Language Learning (CoNLL 2018), pages 211–220 Brussels, Belgium, October 31 - November 1, 2018. c⃝2018 Association for Computational Linguistics 211 Generalizing Procrustes Analysis for Better Bilingual Dictionary Induction Yova Kementchedjhieva3 Sebastian Ruder♠♣ Ryan Cotterell♥ Anders Søgaard3 3University of Copenhagen, Copenhagen, Denmark ♠Insight Research Centre, National University of Ireland, Galway, Ireland ♣Aylien Ltd., Dublin, Ireland ♥University of Cambridge, Cambridge, UK {yova|soegaard}@di.ku.dk,sebastian@ruder.io,ryan.cotterell@gmail.com, Abstract Most recent approaches to bilingual dictionary induction find a linear alignment between the word vector spaces of two languages. We show that projecting the two languages onto a third, latent space, rather than directly onto each other, while equivalent in terms of ex- pressivity, makes it easier to learn approximate alignments. Our modified approach also al- lows for supporting languages to be included in the alignment process, to obtain an even bet- ter performance in low resource settings. 1 Introduction Several papers recently demonstrated the poten- tial of very weakly supervised or entirely unsuper- vised approaches to bilingual dictionary induction (BDI) (Barone, 2016; Artetxe et al., 2017; Zhang et al., 2017; Conneau et al., 2018; Søgaard et al., 2018), the task of identifying translational equiva- lents across two languages. These approaches cast BDI as a problem of aligning monolingual word embeddings. Pairs of monolingual word vector spaces can be aligned without any explicit cross- lingual supervision, solely based on their distri- butional properties (for an adversarial approach, see Conneau et al. (2018)). Alternatively, weak supervision can be provided in the form of nu- merals (Artetxe et al., 2017) or identically spelled words (Søgaard et al., 2018). Successful unsu- pervised or weakly supervised alignment of word vector spaces would remove much of the data bot- tleneck for machine translation and push horizons for cross-lingual learning (Ruder et al., 2018). In addition to an unsupervised approach to aligning monolingual word embedding spaces with adversarial training, Conneau et al. (2018) present a supervised alignment algorithm that as- sumes a gold-standard seed dictionary and per- forms Procrustes Analysis (Sch¨onemann, 1966). Søgaard et al. (2018) show that this approach, weakly supervised with a dictionary seed of cross- lingual homographs, i.e. words with identical spelling across source and target language, is su- perior to the completely unsupervised approach. We therefore focus on weakly-supervised Pro- crustes Analysis (PA) for BDI here. The implementation of PA in Conneau et al. (2018) yields notable improvements over earlier work on BDI, even though it learns a simple lin- ear transform of the source language space into the target language space. Seminal work in supervised alignment of word vector spaces indeed reported superior performance with linear models as com- pared to non-linear neural approaches (Mikolov et al., 2013). The relative success of the simple linear approach can be explained in terms of iso- morphism across monolingual semantic spaces,1 an idea that receives support from cognitive sci- ence (Youn et al., 1999). Word vector spaces are not perfectly isomorphic, however, as shown by Søgaard et al. (2018), who use a Laplacian graph similarity …

Content truncated to 500 words