Beyond Left-to-Right: Multiple Decomposition Structures for SMT

Proceedings of NAACL-HLT 2013, pages 12–21, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics Beyond Left-to-Right: Multiple Decomposition Structures for SMT Hui Zhang∗ USC/ISI Los Angeles, CA 90089 hzhang@isi.edu Kristina Toutanova Microsoft Research Redmond, WA 98502 kristout@microsoft.com Chris Quirk Microsoft Research Redmond, WA 98502 chrisq@microsoft.com Jianfeng Gao Microsoft Research Redmond, WA 98502 jfgao@microsoft.com Abstract Standard phrase-based translation models do not explicitly model context dependence be- tween translation units. As a result, they rely on large phrase pairs and target language mod- els to recover contextual effects in translation. In this work, we explore n-gram models over Minimal Translation Units (MTUs) to explic- itly capture contextual dependencies across phrase boundaries in the channel model. As there is no single best direction in which con- textual information should flow, we explore multiple decomposition structures as well as dynamic bidirectional decomposition. The resulting models are evaluated in an intrin- sic task of lexical selection for MT as well as a full MT system, through n-best rerank- ing. These experiments demonstrate that ad- ditional contextual modeling does indeed ben- efit a phrase-based system and that the direc- tion of conditioning is important. Integrating multiple conditioning orders provides consis- tent benefit, and the most important directions differ by language pair. 1 Introduction The translation procedure of a classical phrase- based translation model (Koehn et al., 2003) first di- vides the input sentence into a sequence of phrases, translates each phrase, explores reorderings of these translations, and then scores the resulting candi- dates with a linear combination of models. Conven- tional models include phrase-based channel models that effectively model each phrase as a large uni- gram, reordering models, and target language mod- els. Of these models, only the target language model ∗This research was conducted during the author’s internship at Microsoft Research (and, to some weak extent, the lexicalized reordering model) captures some lexical dependencies that span phrase boundaries, though it is not able to model in- formation from the source side. Larger phrases cap- ture more contextual dependencies within a phrase, but individual phrases are still translated almost in- dependently. To address this limitation, several researchers have proposed bilingual n-gram Markov models (Marino et al., 2006) to capture contextual depen- dencies between phrase pairs. Much of their work is limited by the requirement “that the source and target side of a tuple of words are synchronized, i.e. that they occur in the same order in their respective languages” (Crego and Yvon, 2010). For language pairs with significant typological di- vergences, such as Chinese-English, it is quite dif- ficult to extract a synchronized sequence of units; in the limit, the smallest synchronized unit may be the whole sentence. Other approaches explore incor- poration into syntax-based MT systems or replacing the phrasal translation system altogether. We investigate the addition of MTUs to a phrasal translation system to improve modeling of con- text and to provide more robust estimation of long phrases. However, in a phrase-based system there is no single synchronized traversal order; instead, we …

Content truncated to 500 words