Non-projective Dependency-based Pre-Reordering with Recurrent Neural Network for Machine Translation

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 846–856, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics Non-projective Dependency-based Pre-Reordering with Recurrent Neural Network for Machine Translation Antonio Valerio Miceli-Barone Universit`a di Pisa Largo B. Pontecorvo, 3 56127 Pisa, Italy miceli@di.unipi.it Giuseppe Attardi Universit`a di Pisa Largo B. Pontecorvo, 3 56127 Pisa, Italy attardi@di.unipi.it Abstract The quality of statistical machine translation performed with phrase based approaches can be increased by permuting the words in the source sentences in an order which resem- bles that of the target language. We propose a class of recurrent neu- ral models which exploit source-side dependency syntax features to re- order the words into a target-like or- der. We evaluate these models on the German-to-English and Italian-to- English language pairs, showing sig- nificant improvements over a phrase- based Moses baseline. We also com- pare with state of the art German-to- English pre-reordering rules, showing that our method obtains similar or bet- ter results. 1 Introduction Statistical machine translation is typically performed using phrase-based systems (Koehn et al., 2007). These systems can usually produce accurate local reordering but they have difficulties dealing with the long-distance reordering that tends to occur between certain language pairs (Birch et al., 2008). The quality of phrase-based machine trans- lation can be improved by reordering the words in each sentence of source-side of the parallel training corpus in a ”target-like” or- der and then applying the same transforma- tion as a pre-processing step to input strings during execution. When the source-side sentences can be ac- curately parsed, pre-reordering can be per- formed using hand-coded rules. This ap- proach has been successfully applied to German-to-English (Collins et al., 2005) and other languages. The main issue with it is that these rules must be designed for each spe- cific language pair, which requires consider- able linguistic expertise. Fully statistical approaches, on the other hand, learn the reordering relation from word alignments. Some of them learn reordering rules on the constituency (Dyer and Resnik, 2010) (Khalilov and Fonollosa, 2011) or pro- jective dependency (Genzel, 2010), (Lerner and Petrov, 2013) parse trees of source sen- tences. The permutations that these meth- ods can learn can be generally non-local (i.e. high distance) on the sentences but lo- cal (parent-child or sibling-sibling swaps) on the parse trees. Moreover, constituency or projective dependency trees may not be the ideal way of representing the syntax of non- analytic languages such as German or Ital- ian, which could be better described using non-projective dependency trees (Bosco and Lombardo, 2004). Other methods, based on recasting reordering as a combinatorial opti- mization problem (Tromble and Eisner, 2009), (Visweswariah et al., 2011), can learn to gen- erate in principle arbitrary permutations, but they can only make use of minimal syntactic information (part-of-speech tags) and there- fore can’t exploit the potentially valuable structural syntactic information provided by a parser. In this work we propose a class of …

Content truncated to 500 words