A Joint Sequence Translation Model with Integrated Reordering

Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 1045–1054, Portland, Oregon, June 19-24, 2011. c⃝2011 Association for Computational Linguistics A Joint Sequence Translation Model with Integrated Reordering Nadir Durrani Helmut Schmid Alexander Fraser Institute for Natural Language Processing University of Stuttgart {durrani,schmid,fraser}@ims.uni-stuttgart.de Abstract We present a novel machine translation model which models translation by a linear sequence of operations. In contrast to the “N-gram” model, this sequence includes not only trans- lation but also reordering operations. Key ideas of our model are (i) a new reordering approach which better restricts the position to which a word or phrase can be moved, and is able to handle short and long distance re- orderings in a unified way, and (ii) a joint sequence model for the translation and re- ordering probabilities which is more flexi- ble than standard phrase-based MT. We ob- serve statistically significant improvements in BLEU over Moses for German-to-English and Spanish-to-English tasks, and comparable re- sults for a French-to-English task. 1 Introduction We present a novel generative model that explains the translation process as a linear sequence of oper- ations which generate a source and target sentence in parallel. Possible operations are (i) generation of a sequence of source and target words (ii) insertion of gaps as explicit target positions for reordering op- erations, and (iii) forward and backward jump oper- ations which do the actual reordering. The probabil- ity of a sequence of operations is defined according to an N-gram model, i.e., the probability of an op- eration depends on the n − 1 preceding operations. Since the translation (generation) and reordering op- erations are coupled in a single generative story, the reordering decisions may depend on preceding translation decisions and translation decisions may depend on preceding reordering decisions. This pro- vides a natural reordering mechanism which is able to deal with local and long-distance reorderings in a consistent way. Our approach can be viewed as an extension of the N-gram SMT approach (Mari˜no et al., 2006) but our model does reordering as an inte- gral part of a generative model. The paper is organized as follows. Section 2 dis- cusses the relation of our work to phrase-based and the N-gram SMT. Section 3 describes our genera- tive story. Section 4 defines the probability model, which is first presented as a generative model, and then shifted to a discriminative framework. Section 5 provides details on the search strategy. Section 6 explains the training process. Section 7 describes the experimental setup and results. Section 8 gives a few examples illustrating different aspects of our model and Section 9 concludes the paper. 2 Motivation and Previous Work 2.1 Relation of our work to PBSMT Phrase-based SMT provides a powerful translation mechanism which learns local reorderings, transla- tion of short idioms, and the insertion and deletion of words sensitive to local context. However, PBSMT also has some drawbacks. (i) Dependencies across phrases are not directly represented in the translation model. (ii) Discontinuous phrases cannot be used. …

Content truncated to 500 words