A Neural Reordering Model for Phrase-based Translation

Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1897–1907, Dublin, Ireland, August 23-29 2014. A Neural Reordering Model for Phrase-based Translation Peng Li† Yang Liu† Maosong Sun† Tatsuya Izuha‡ Dakun Zhang∗ †State Key Laboratory of Intelligent Technology and Systems Tsinghua National Laboratory for Information Science and Technology Department of Computer Sci. and Tech., Tsinghua University, Beijing, China pengli09@gmail.com, {liuyang2011,sms}@tsinghua.edu.cn ‡Toshiba Corporation Corporate Research & Development Center tatsuya.izuha@toshiba.co.jp ∗Toshiba (China) R&D Center zhangdakun@toshiba.com.cn Abstract While lexicalized reordering models have been widely used in phrase-based translation systems, they suffer from three drawbacks: context insensitivity, ambiguity, and sparsity. We propose a neural reordering model that conditions reordering probabilities on the words of both the current and previous phrase pairs. Including the words of previous phrase pairs significantly improves context sensitivity and reduces reordering ambiguity. To alleviate the data sparsity problem, we build one classifier for all phrase pairs, which are represented as continuous space vectors. Ex- periments on the NIST Chinese-English datasets show that our neural reordering model achieves significant improvements over state-of-the-art lexicalized reordering models. 1 Introduction Reordering plays a crucial role in phrase-based translation (Koehn et al., 2003; Och and Ney, 2004). While local reordering can be directly memorized in phrases, modeling reordering at a phrase level still remains a major challenge: it can be cast as a travelling salesman problem and proves to be NP-complete (Knight, 1999; Zaslavskiy et al., 2009). The past decade has witnessed the rapid development of phrase reordering models (e.g., (Och et al., 2004; Tillman, 2004; Zens et al., 2004; Xiong et al., 2006; Al-Onaizan and Papineni, 2006; Koehn et al., 2007; Galley and Manning, 2008; Feng et al., 2010; Green et al., 2010; Bisazza and Federico, 2012; Cherry, 2013), just to name a few). Among them, lexicalized reordering models (Tillman, 2004; Koehn et al., 2007; Galley and Manning, 2008) have been widely used in practical phrase-based systems. Un- like the distance-based reordering model (Koehn et al., 2003) that only penalizes phrase displacements in terms of the degree of nonmonotonicity, lexicalized reordering models introduce reordering probabil- ities conditioned on the words of each phrase pair. They often distinguish between three orientations with respect to the previous phrase pair: monotone, swap, and discontinuous. As lexicalized reordering models capture the phenomenon that some words are far more likely to be displaced than others, they outperform unlexicalized reordering models substantially. Despite their apparent success in statistical machine translation, lexicalized reordering models suffer from the following three drawbacks: 1. Context insensitivity. Lexicalized reordering models determine the orientations only depending on the words of current phrase pairs. In fact, a phrase pair usually has different orientations in different contexts. It is important to include more contexts to improve the expressive power of reordering models. 2. Ambiguity. Short phrase pairs, which are observed in the training data more frequently, usually have multiple orientations. We observe that about 92.4% of one-word Chinese-English phrase pairs are ambiguous. This makes it hard to decide which orientation should be properly …

Content truncated to 500 words