Bilingual Lexicon Induction via Unsupervised Bitext Construction and Word Alignment

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 813–826 August 1–6, 2021. ©2021 Association for Computational Linguistics 813 Bilingual Lexicon Induction via Unsupervised Bitext Construction and Word Alignment Haoyue Shi ∗ TTI-Chicago freda@ttic.edu Luke Zettlemoyer University of Washington Facebook AI Research lsz@fb.com Sida I. Wang Facebook AI Research sida@fb.com Abstract Bilingual lexicons map words in one language to their translations in another, and are typi- cally induced by learning linear projections to align monolingual word embedding spaces. In this paper, we show it is possible to produce much higher quality lexicons with methods that combine (1) unsupervised bitext mining and (2) unsupervised word alignment. Directly applying a pipeline that uses recent algorithms for both subproblems significantly improves induced lexicon quality and further gains are possible by learning to filter the resulting lex- ical entries, with both unsupervised and semi- supervised schemes. Our final model outper- forms the state of the art on the BUCC 2020 shared task by 14 F1 points averaged over 12 language pairs, while also providing a more in- terpretable approach that allows for rich rea- soning of word meaning in context. Further analysis of our output and the standard refer- ence lexicons suggests they are of comparable quality, and new benchmarks may be needed to measure further progress on this task.1 1 Introduction Bilingual lexicons map words in one language to their translations in another, and can be automati- cally induced by learning linear projections to align monolingual word embedding spaces (Artetxe et al., 2016; Smith et al., 2017; Lample et al., 2018, inter alia). Although very successful in practice, the linear nature of these methods encodes unrealis- tic simplifying assumptions (e.g. all translations of a word have similar embeddings). In this paper, we show it is possible to produce much higher quality lexicons without these restrictions by introducing new methods that combine (1) unsupervised bitext mining and (2) unsupervised word alignment. ∗Work done during internship at Facebook AI Research. 1Code is publicly available at https://github.com/ facebookresearch/bitext-lexind. We show that simply pipelining recent algo- rithms for unsupervised bitext mining (Tran et al., 2020) and unsupervised word alignment (Sabet et al., 2020) significantly improves bilingual lexi- con induction (BLI) quality, and that further gains are possible by learning to filter the resulting lexi- cal entries. Improving on a recent method for doing BLI via unsupervised machine translation (Artetxe et al., 2019), we show that unsupervised mining produces better bitext for lexicon induction than translation, especially for less frequent words. These core contributions are established by sys- tematic experiments in the class of bitext construc- tion and alignment methods (Figure 1). Our full induction algorithm filters the lexicon found via the initial unsupervised pipeline. The filtering can be either fully unsupervised or weakly-supervised: for the former, we filter using simple heuristics and global statistics; for the latter, we train a multi-layer perceptron (MLP) to predict the probability of a word pair being in …

Content truncated to 500 words