Learning Translations via Matrix Completion

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1452–1463 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Learning Translations via Matrix Completion Derry Wijaya1, Brendan Callahan1, John Hewitt1, Jie Gao1, Xiao Ling1, Marianna Apidianaki12 and Chris Callison-Burch1 1Computer and Information Science Department, University of Pennsylvania 2LIMSI, CNRS, Universit´e Paris-Saclay, 91403 Orsay derry@seas.upenn.edu Abstract Bilingual Lexicon Induction is the task of learning word translations without bilin- gual parallel corpora. We model this task as a matrix completion problem, and present an effective and extendable frame- work for completing the matrix. This method harnesses diverse bilingual and monolingual signals, each of which may be incomplete or noisy. Our model achieves state-of-the-art performance for both high and low resource languages. 1 Introduction Machine translation (MT) models typically re- quire large, sentence-aligned bilingual texts to learn good translation models (Wu et al., 2016; Sennrich et al., 2016a; Koehn et al., 2003). How- ever, for many language pairs, such parallel texts may only be available in limited quantities, which is problematic. Alignments at the word- or subword- levels (Sennrich et al., 2016b) can be in- accurate in the limited parallel texts, which can in turn lead to inaccurate translations. Due to the low quantity and thus coverage of the texts, there may still be “out-of-vocabulary” words encountered at run-time. The Bilingual Lexicon Induction (BLI) task (Rapp, 1995), which learns word translations from monolingual or comparable corpora, is an at- tempt to alleviate this problem. The goal is to use plentiful, more easily obtainable, monolingual or comparable data to infer word translations and re- duce the need for parallel data to learn good trans- lation models. The word translations obtained by BLI can, for example, be used to augment MT systems and improve alignment accuracy, cover- age, and translation quality (Gulcehre et al., 2016; Callison-Burch et al., 2006; Daum´e and Jagarla- mudi, 2011). fridge house fridge house sleep sleep kulkas koki tidur hijzelf koelkast kokkien EN NL fridge house fridge house sleep sleep kulkas koki tidur kast koelkast kokkien EN ID EN ID Figure 1: Our framework allows us to use a di- verse range of signals to learn translations, in- cluding incomplete bilingual dictionaries, infor- mation from related languages (like Indonesian loan words from Dutch shown here), word embed- dings, and even visual similarity cues. Previous research has explored different sources for estimating translation equivalence from mono- lingual corpora (Schafer and Yarowsky, 2002; Klementiev and Roth, 2006; Irvine and Callison- Burch, 2013, 2017). These monolingual signals, when combined in a supervised model, can en- hance end-to-end MT for low resource languages (Klementiev et al., 2012a; Irvine and Callison- Burch, 2016). More recently, similarities between words in different languages have been approxi- mated by constructing a shared bilingual word em- bedding space with different forms of bilingual su- pervision (Upadhyay et al., 2016). We present a framework for learning transla- tions by combining diverse signals of translation that are each potentially sparse or noisy. We use matrix factorization (MF), …

Content truncated to 500 words