RAPO: An Adaptive Ranking Paradigm for Bilingual Lexicon Induction

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8870 - 8883 December 7-11, 2022 ©2022 Association for Computational Linguistics RAPO: An Adaptive Ranking Paradigm for Bilingual Lexicon Induction Zhoujin Tian∗, Chaozhuo Li†, Shuo Ren, Zhiqiang Zuo, Zengxuan Wen, Xinyue Hu Xiao Han, Haizhen Huang, Denvy Deng, Qi Zhang, Xing Xie Microsoft deritt7@gmail.com, {cli,renshuo,zhiqzuo,zewen,xinyuehu}@microsoft.com {xiaoha,hhuang,dedeng,qizhang,xingx}@microsoft.com Abstract Bilingual lexicon induction induces the word translations by aligning independently trained word embeddings in two languages. Existing approaches generally focus on minimizing the distances between words in the aligned pairs, while suffering from low discriminative capa- bility to distinguish the relative orders between positive and negative candidates. In addition, the mapping function is globally shared by all words, whose performance might be hindered by the deviations in the distributions of dif- ferent languages. In this work, we propose a novel ranking-oriented induction model RAPO to learn personalized mapping function for each word. RAPO is capable of enjoying the merits from the unique characteristics of a single word and the cross-language isomorphism simultane- ously. Extensive experimental results on public datasets including both rich-resource and low- resource languages demonstrate the superiority of our proposal. Our code is publicly available in https://github.com/Jlfj345wf/RAPO. 1 Introduction Bilingual lexicon induction (BLI) aims at inducing the word translations across two languages based on the monolingual corpora, which is capable of transferring valuable semantic knowledge between different languages, spawning a myriad of NLP tasks such as machine translation (Lample et al., 2018; Artetxe et al., 2018c), semantic parsing (Xiao and Guo, 2014), and document classification (Kle- mentiev et al., 2012). The nucleus of BLI is learn- ing a desirable mapping function to align two sets of independently trained monolingual word embed- dings (Mikolov et al., 2013; Ruder et al., 2019; Glavaš et al., 2019). Mikolov et al. (2013) empiri- cally observe that the linear projections are superior to their non-linear counterparts due to the isomor- phism across different embedding spaces. Sub- ∗ Work is done during internship at Microsoft. † Corresponding author and equal contribution. sequent improvements are successively proposed to advance BLI task by imposing orthogonal con- straints (Xing et al., 2015; Conneau et al., 2018), normalizing the embeddings (Artetxe et al., 2016, 2018a; Zhang et al., 2019), reducing the noises (Artetxe et al., 2018b; Yehezkel Lubin et al., 2019), relaxing the hypothesis of isomorphism (Søgaard et al., 2018; Patra et al., 2019), and iteratively re- fining the seed dictionary (Zhao et al., 2020). Existing methods (Artetxe et al., 2016, 2018a; Jawanpuria et al., 2020) usually aim at minimizing the distance between the word from the source lan- guage and its aligned word in the target language (e.g., crow and cuervo in Figure 1). However, BLI is essentially a ranking-oriented task because for each source word, we expect to select its top k high- confidence target candidates. Namely, a desirable BLI model should also be capable of distinguishing the relative orders between the positive and nega- tive candidates (e.g., crow and curevo should be distributed closer than …

Content truncated to 500 words