Dependency-Based Bilingual Language Models for Reordering in Statistical Machine Translation

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1689–1700, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Dependency-Based Bilingual Language Models for Reordering in Statistical Machine Translation Ekaterina Garmash and Christof Monz Informatics Institute, University of Amsterdam Science Park 904, 1098 XH Amsterdam, The Netherlands {e.garmash,c.monz}@uva.nl Abstract This paper presents a novel approach to improve reordering in phrase-based ma- chine translation by using richer, syntac- tic representations of units of bilingual language models (BiLMs). Our method to include syntactic information is simple in implementation and requires minimal changes in the decoding algorithm. The approach is evaluated in a series of Arabic- English and Chinese-English translation experiments. The best models demon- strate significant improvements in BLEU and TER over the phrase-based baseline, as well as over the lexicalized BiLM by Niehues et al. (2011). Further improve- ments of up to 0.45 BLEU for Arabic- English and up to 0.59 BLEU for Chinese- English are obtained by combining our de- pendency BiLM with a lexicalized BiLM. An improvement of 0.98 BLEU is ob- tained for Chinese-English in the setting of an increased distortion limit. 1 Introduction In statistical machine translation (SMT) reorder- ing (also called distortion) refers to the order in which source words are translated to generate the translation in the target language. Word orders can differ significantly across languages. For in- stance, Arabic declarative sentences can be verb- initial, while the corresponding English translation should realize the verb after the subject, hence re- quiring a reordering. Determining the correct re- ordering during decoding is a major challenge for SMT. This problem has received a lot of attention in the literature (see, e.g., Tillmann (2004), Zens and Ney (2003), Al-Onaizan and Papineni (2006)), as choosing the correct reordering improves read- ability of the translation and can have a substan- tial impact on translation quality (Birch, 2011). In this paper, we only consider those approaches that include a reordering feature function into the log- linear interpolation used during decoding. The simplest reordering model is linear distor- tion (Koehn et al., 2003) which scores the distance between phrases translated at steps t and t + 1 of the derivation. This model ignores any contex- tual information, as the distance between trans- lated phrases is its only parameter. Lexical dis- tortion modeling (Tillmann, 2004) conditions re- ordering probabilities on the phrase pairs trans- lated at the current and previous steps. Unlike linear distortion, it characterizes reordering not in terms of distance but type: monotone, swap, or discontinuous. In this paper, we base our approach to reorder- ing on bilingual language models (Marino et al., 2006; Niehues et al., 2011). Instead of directly characterizing reordering, they model sequences of elementary translation events as a Markov pro- cess.1 Originally, Marino et al. (2006) used this kind of model as the translation model, while more recently it has been used as an additional model in PBSMT systems (Niehues et al., 2011). We adopt and generalize the approach of Niehues …

Content truncated to 500 words