Beyond Parallel Data: Joint Word Alignment and Decipherment Improves Machine Translation

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 557–565, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Beyond Parallel Data: Joint Word Alignment and Decipherment Improves Machine Translation Qing Dou , Ashish Vaswani, and Kevin Knight Information Sciences Institute Department of Computer Science University of Southern California {qdou,avaswani,knight}@isi.edu Abstract Inspired by previous work, where decipher- ment is used to improve machine translation, we propose a new idea to combine word align- ment and decipherment into a single learning process. We use EM to estimate the model pa- rameters, not only to maximize the probabil- ity of parallel corpus, but also the monolingual corpus. We apply our approach to improve Malagasy-English machine translation, where only a small amount of parallel data is avail- able. In our experiments, we observe gains of 0.9 to 2.1 Bleuover a strong baseline. 1 Introduction State-of-the-art machine translation (MT) systems ap- ply statistical techniques to learn translation rules au- tomatically from parallel data. However, this reliance on parallel data seriously limits the scope of MT ap- plication in the real world, as for many languages and domains, there is not enough parallel data to train a de- cent quality MT system. However, compared with parallel data, there are much larger amounts of non parallel data. The abil- ity to learn a translation lexicon or even build a ma- chine translation system using monolingual data helps address the problems of insufficient parallel data. Ravi and Knight (2011) are among the first to learn a full MT system using only non parallel data through deci- pherment. However, the performance of such systems is much lower compared with those trained with par- allel data. In another work, Klementiev et al. (2012) show that, given a phrase table, it is possible to esti- mate parameters for a phrase-based MT system from non parallel data. Given that we often have some parallel data, it is more practical to improve a translation system trained on parallel data by using additional non parallel data. Rapp (1995) shows that with a seed lexicon, it is possi- ble to induce new word level translations from non par- allel data. Motivated by the idea that a translation lexi- con induced from non parallel data can be used to trans- late out of vocabulary words (OOV), a variety of prior research has tried to build a translation lexicon from non parallel or comparable data (Fung and Yee, 1998; Koehn and Knight, 2002; Haghighi et al., 2008; Garera Figure 1: Combine word alignment and decipherment into a single learning process. et al., 2009; Bergsma and Van Durme, 2011; Daum´e and Jagarlamudi, 2011; Irvine and Callison-Burch, 2013b; Irvine and Callison-Burch, 2013a; Irvine et al., 2013). Lately, there has been increasing interest in learn- ing translation lexicons from non parallel data with de- cipherment techniques (Ravi and Knight, 2011; Dou and Knight, 2012; Nuhn et al., 2012; Dou and Knight, 2013). Decipherment views one language as a cipher for another and learns …

Content truncated to 500 words