An Alignment-Based Approach to Semi-Supervised Bilingual Lexicon Induction with Small Parallel Corpora

An Alignment-Based Approach to Semi-Supervised Bilingual Lexicon Induction with Small Parallel Corpora Kelly Marchisio kmarc@jhu.edu Conghao Xiong cxiong5@jhu.edu Philipp Koehn phi@jhu.edu Center for Language and Speech Processing, Johns Hopkins University, Baltimore, MD, 21218, USA Abstract Aimed at generating a seed lexicon for use in downstream natural language tasks, unsuper- vised methods for bilingual lexicon induction have received much attention in the academic literature recently. While interesting, fully unsupervised settings are unrealistic; small amounts of bilingual data are usually available due to the existence of massively multilingual parallel corpora, or linguists can create small amounts of parallel data. In this work, we demonstrate an effective bootstrapping approach for semi-supervised bilingual lexicon induction that cap- italizes upon the complementary strengths of two disparate methods for inducing bilingual lexicons. Whereas statistical methods are highly effective at inducing correct translation pairs for words frequently occurring in a parallel corpus, monolingual embedding spaces have the advantage of having been trained on large amounts of data, and therefore may induce accurate translations for words absent from the small corpus. By combining these relative strengths, our method achieves state-of-the-art results on 3 of 4 language pairs in the challenging VecMap test set using minimal amounts of parallel data and without the need for a translation dic- tionary. We release our implementation at https://github.com/kellymarchisio/ align-semisup-bli. 1 Introduction Unsupervised methods for machine translation (MT) and bilingual lexicon induction (BLI) have received considerable attention in recent years, showing impressive performance without bilin- gual data for supervision. While academically interesting, small amounts of supervised data can almost always help model performance. The typical use case for unsupervised BLI is to provide initial synthetic training data for a traditional supervised setup where no parallel bitext exists, such as for MT or cross-lingual information retrieval. A starting lexicon is induced in an unsupervised manner, and then serves as initial training data to the supervised model. Practically, however, one struggles to identify a scenario where one would truly fail to have any parallel text whatesoever from which to gain some supervision. The Christian Bible, for instance, is translated into over 1600 world lan- guages, providing multi-way parallel data for many of the world’s languages that are typically considered “low-resource” (McCarthy et al., 2020). Human translators can also create a small translation corpus or seed dictionary. The practical necessity of fully unsupervised scenarios for BLI or MT therefore becomes hard to imagine. Proceedings of the 18th Biennial Machine Translation Summit Virtual USA, August 16 - 20, 2021, Volume 1: MT Research Track Page 293 Statistical translation/alignment models are very proficient at inducing bilingual lexicons from small amounts of parallel data. Particularly when words occur frequently in the corpus, statistical models easily recover the translation. At the same time, however, the number of seed translation pairs possible to extract is limited by the vocabulary of the parallel corpus. We address a more realistic scenario: there is ample monolingual data and a small parallel corpus. We combine the strengths of statistical alignment and unsupervised mapping methods and …

Content truncated to 500 words