A Simple and Effective Approach to Robust Unsupervised Bilingual Dictionary Induction

Proceedings of the 28th International Conference on Computational Linguistics, pages 5990–6001 Barcelona, Spain (Online), December 8-13, 2020 5990 A Simple and Effective Approach to Robust Unsupervised Bilingual Dictionary Induction Yanyang Li1∗, Yingfeng Luo1∗, Ye Lin1, Quan Du1, Huizhen Wang1, Shujian Huang3,4, Tong Xiao1,2† and Jingbo Zhu1,2 1Natural Language Processing Lab., Northeastern University, Shenyang, China 2NiuTrans Research, Shenyang, China 3National Key Laboratory for Novel Software Technology, Nanjing, China 4Nanjing University, Nanjing, China blamedrlee@outlook.com, 1971646@stu.neu.edu.cn, {linye2015, duquanneu}@outlook.com, huangsj@nju.edu.cn, {wanghuizhen, xiaotong, zhujingbo}@mail.neu.edu.cn Abstract Unsupervised Bilingual Dictionary Induction methods based on the initialization and the self- learning have achieved great success in similar language pairs, e.g., English-Spanish. But they still fail and have an accuracy of 0% in many distant language pairs, e.g., English-Japanese. In this work, we show that this failure results from the gap between the actual initialization performance and the minimum initialization performance for the self-learning to succeed. We propose Iterative Dimension Reduction to bridge this gap. Our experiments show that this simple method does not hamper the performance of similar language pairs and achieves an accuracy of 13.64∼55.53% between English and four distant languages, i.e., Chinese, Japanese, Vietnamese and Thai. 1 Introduction Unsupervised Bilingual Dictionary Induction (UBDI) is a task that aims to find the word translations given the monolingual word embeddings of two languages. Recent UBDI methods have shown promising results on similar language pairs such as English-Spanish (Artetxe et al., 2017; Lample et al., 2018; Zhou et al., 2019; Hartmann et al., 2019; Ren et al., 2020). These methods are mostly based on the initialization and the self-learning. The initialization first constructs a dictionary from the word embeddings, then the self-learning starts with this dictionary and alternates between refining the source-target word embedding mapping and inducing a new dictionary with this mapping. Despite the success of UBDI, recent work has questioned the robustness of UBDI methods on distant language pairs (Søgaard et al., 2018; Vuli´c et al., 2019; Glavas et al., 2019), e.g., English-Japanese. They show that even for the most robust system VecMap (Artetxe et al., 2018), it still fails and has an accuracy of 0% on 87 out of 210 distant language pairs (Vuli´c et al., 2019). To be consistent with Artetxe et al. (2018), we define a system ‘succeeds’ when it has an accuracy above 5% and ‘fails’ otherwise. Previous work has investigated how different properties of languages have an impact on UBDI perfor- mance (Søgaard et al., 2018). In this paper, we take a step further to inspect which part of VecMap breaks down. With a novel similarity metric to evaluate the initialization performance, we observe a gap between the actual initialization performance and the minimum initialization performance for the self-learning to succeed in distant language pairs. We find that the dimension reduction approach is very effective in bridging this gap. Therefore, we propose Iterative Dimension Reduction (IDR) to improve the robustness of VecMap and avoid performance loss due to dimension reduction. IDR first reduces the dimension of word embeddings and performs unsupervised learning …

Content truncated to 500 words