Maximum Entropy Based Phrase Reordering Model for Statistical Machine Translation

Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 521–528, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Maximum Entropy Based Phrase Reordering Model for Statistical Machine Translation Deyi Xiong Institute of Computing Technology Chinese Academy of Sciences Beijing, China, 100080 Graduate School of Chinese Academy of Sciences dyxiong@ict.ac.cn Qun Liu and Shouxun Lin Institute of Computing Technology Chinese Academy of Sciences Beijing, China, 100080 {liuqun, sxlin}@ict.ac.cn Abstract We propose a novel reordering model for phrase-based statistical machine transla- tion (SMT) that uses a maximum entropy (MaxEnt) model to predicate reorderings of neighbor blocks (phrase pairs). The model provides content-dependent, hier- archical phrasal reordering with general- ization based on features automatically learned from a real-world bitext. We present an algorithm to extract all reorder- ing events of neighbor blocks from bilin- gual data. In our experiments on Chinese- to-English translation, this MaxEnt-based reordering model obtains significant im- provements in BLEU score on the NIST MT-05 and IWSLT-04 tasks. 1 Introduction Phrase reordering is of great importance for phrase-based SMT systems and becoming an ac- tive area of research recently. Compared with word-based SMT systems, phrase-based systems can easily address reorderings of words within phrases. However, at the phrase level, reordering is still a computationally expensive problem just like reordering at the word level (Knight, 1999). Many systems use very simple models to re- order phrases 1. One is distortion model (Och and Ney, 2004; Koehn et al., 2003) which penal- izes translations according to their jump distance instead of their content. For example, if N words are skipped, a penalty of N will be paid regard- less of which words are reordered. This model takes the risk of penalizing long distance jumps 1In this paper, we focus our discussions on phrases that are not necessarily aligned to syntactic constituent boundary. which are common between two languages with very different orders. Another simple model is flat reordering model (Wu, 1996; Zens et al., 2004; Kumar et al., 2005) which is not content depen- dent either. Flat model assigns constant probabili- ties for monotone order and non-monotone order. The two probabilities can be set to prefer mono- tone or non-monotone orientations depending on the language pairs. In view of content-independency of the dis- tortion and flat reordering models, several re- searchers (Och et al., 2004; Tillmann, 2004; Ku- mar et al., 2005; Koehn et al., 2005) proposed a more powerful model called lexicalized reorder- ing model that is phrase dependent. Lexicalized reordering model learns local orientations (mono- tone or non-monotone) with probabilities for each bilingual phrase from training data. During de- coding, the model attempts to finding a Viterbi lo- cal orientation sequence. Performance gains have been reported for systems with lexicalized reorder- ing model. However, since reorderings are re- lated to concrete phrases, researchers have to de- sign their systems carefully in order not to cause other problems, e.g. the data sparseness problem. Another smart reordering model was proposed by Chiang (2005). In …

Content truncated to 500 words