Wider Context by Using Bilingual Language Models in Machine Translation

Proceedings of the 6th Workshop on Statistical Machine Translation, pages 198–206, Edinburgh, Scotland, UK, July 30–31, 2011. c⃝2011 Association for Computational Linguistics Wider Context by Using Bilingual Language Models in Machine Translation Jan Niehues1, Teresa Herrmann1, Stephan Vogel2 and Alex Waibel1,2 1Institute for Anthropomatics, KIT - Karlsruhe Institute of Technology, Germany 2 Language Techonolgies Institute, Carnegie Mellon University, USA 1firstname.lastname@kit.edu 2lastname@cs.cmu.edu Abstract In past Evaluations for Machine Translation of European Languages, it could be shown that the translation performance of SMT systems can be increased by integrating a bilingual lan- guage model into a phrase-based SMT system. In the bilingual language model, target words with their aligned source words build the to- kens of an n-gram based language model. We analyzed the effect of bilingual language mod- els and show where they could help to bet- ter model the translation process. We could show improvements of translation quality on German-to-English and Arabic-to-English. In addition, for the Arabic-to-English task, train- ing an extra bilingual language model on the POS tags instead of the surface word forms led to further improvements. 1 Introduction In many state-of-the art SMT systems, the phrase- based (Koehn et al., 2003) approach is used. In this approach, instead of building the translation by translating word by word, sequences of source and target words, so-called phrase pairs, are used as the basic translation unit. A table of correspondences between source and target phrases forms the transla- tion model in this approach. Target language fluency is modeled by a language model storing monolin- gual n-gram occurrences. A log-linear combination of these main models as well as additional features is used to score the different translation hypotheses. Then the decoder searches for the translation with the highest score. A different approach to SMT is to use a stochas- tic finite state transducer based on bilingual n- grams (Casacuberta and Vidal, 2004). This ap- proach was for example successfully applied by Al- lauzen et al. (2010) on the French-English trans- lation task. In this so-called n-gram approach the translation model is trained by using an n-gram lan- guage model of pairs of source and target words, called tuples. While the phrase-based approach cap- tures only bilingual context within the phrase pairs, in the n-gram approach the n-gram model trained on the tuples is used to capture bilingual context be- tween the tuples. As in the phrase-based approach, the translation model can also be combined with ad- ditional models like, for example, language models using log-linear combination. Inspired by the n-gram-based approach, we in- troduce a bilingual language model that extends the translation model of the phrase-based SMT ap- proach by providing bilingual word context. In ad- dition to the bilingual word context, this approach enables us also to integrate a bilingual context based on part of speech (POS) into the translation model. When using phrase pairs it is complicated to use different kinds of bilingual contexts, since the con- text of the POS-based phrase pairs should be bigger than the word-based …

Content truncated to 500 words