Semi-Supervised Bilingual Lexicon Induction with Two-way Interaction

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 2973–2984, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 2973 Semi-Supervised Bilingual Lexicon Induction with Two-way Interaction Xu Zhao1, Zihao Wang2, Hao Wu3, Zhang Yong1 † 1BNRist, Department of Computer Science and Technology, RIIT, Institute of Internet Industry, Tsinghua University, Beijing, China 2Department of CSE, HKUST, Hong Kong SAR, China 3Department of Mathematical Sciences, Tsinghua University, Beijing, China zhaoxu18@mails.tsinghua.edu.cn zwangggc@cse.ust.hk {hwu,zhangyong05}@tsinghua.edu.cn Abstract Semi-supervision is a promising paradigm for Bilingual Lexicon Induction (BLI) with lim- ited annotations. However, previous semi- supervised methods do not fully utilize the knowledge hidden in annotated and non- annotated data, which hinders further improve- ment of their performance. In this paper, we propose a new semi-supervised BLI frame- work to encourage the interaction between the supervised signal and unsupervised alignment. We design two message-passing mechanisms to transfer knowledge between annotated and non-annotated data, named prior optimal trans- port and bi-directional lexicon update respec- tively. Then, we perform semi-supervised learning based on a cyclic or a parallel pa- rameter feeding routine to update our models. Our framework is a general framework that can incorporate any supervised and unsuper- vised BLI methods based on optimal transport. Experimental results on MUSE and VecMap datasets show significant improvement of our models. Ablation study also proves that the two-way interaction between the supervised signal and unsupervised alignment accounts for the gain of the overall performance. Re- sults on distant language pairs further illustrate the advantage and robustness of our proposed method. 1 Introduction Bilingual Lexicon Induction (BLI) is of huge inter- est to the research frontier. BLI methods learn cross-lingual word embeddings from separately trained monolingual embeddings. BLI is believed to be a promising way to transfer semantic informa- tion between different languages, and spawns lots of NLP applications like machine translation (Lam- ple et al., 2018b; Artetxe et al., 2018b), Part Of Speech (POS) tagging (Gaddy et al., 2016), parsing †Yong Zhang is the corresponding author. (Xiao and Guo, 2014), and document classification (Klementiev et al., 2012). The key step of BLI is to learn a transfor- mation between monolingual word embedding spaces (Ruder et al., 2019), which could be fur- ther used for translation retrieval or cross-lingual analogy tasks. However, it is hard to obtain the high quality transformation with low supervision signals, i.e. with limited annotated lexicon. Thus, some semi-supervised BLI methods (Artetxe et al., 2017; Patra et al., 2019) are proposed to make use of annotated and non-annotated data. Artetxe et al. (2017) bootstrapped the supervised lexicon to en- hance the supervision but ignored the knowledge in non-annotated data. Meanwhile, Patra et al. (2019) combined the unsupervised BLI loss that captured the structural similarity in word embeddings (Lam- ple et al., 2018a) with the supervised loss (Joulin et al., 2018). However, this loss combination still performed poorly since the bad supervised opti- mization under limited annotations, see the Exper- iment part for details. As a result, existing semi- supervised BLI methods suffer from low …

Content truncated to 500 words