Fast and Accurate Preordering for SMT using Neural Networks

Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 1012–1017, Denver, Colorado, May 31 – June 5, 2015. c⃝2015 Association for Computational Linguistics Fast and Accurate Preordering for SMT using Neural Networks Adri`a de Gispert Gonzalo Iglesias Bill Byrne SDL Research East Road, Cambridge CB1 1BH, U.K. {agispert|giglesias|bbyrne}@sdl.com Abstract We propose the use of neural networks to model source-side preordering for faster and better statistical machine translation. The neu- ral network trains a logistic regression model to predict whether two sibling nodes of the source-side parse tree should be swapped in order to obtain a more monotonic parallel corpus, based on samples extracted from the word-aligned parallel corpus. For multiple language pairs and domains, we show that this yields the best reordering performance against other state-of-the-art techniques, resulting in improved translation quality and very fast de- coding. 1 Introduction Preordering is a pre-processing task in translation that aims to reorder the source sentence so that it best resembles the order of the target sentence. If done correctly, it has a doubly beneficial effect: it allows a better estimation of word alignment and translation models which results in higher transla- tion quality for distant language pairs, and it speeds up decoding enormously as less word movement is required. Preordering schemes can be automatically learnt from source-side parsed, word-aligned parallel cor- pora. Recently Jehl et al (2014) described a scheme based on a feature-rich logistic regression model that predicts whether a pair of sibling nodes in the source-side dependency tree need to be permuted. Based on the node-pair swapping probability pre- dictions of this model, a branch-and-bound search returns the best ordering of nodes in the tree. We propose using a neural network (NN) to es- timate this node-swapping probability. We find that this straightforward change to their scheme has mul- tiple advantages: 1. The superior modeling capabilities of NNs achieve better performance at preordering and overall translation quality when using the same set of features. 2. There is no need to manually define which feature combinations are to be considered in training. 3. Preordering is even faster as a result of the previ- ous point. Our results in translating from English to Japanese, Korean, Chinese, Arabic and Hindi support these findings by comparing against two other preordering schemes. 1.1 Related Work There is a strong research and commercial in- terest in preordering, as reflected by the exten- sive previous work on the subject (Collins et al., 2005; Xu et al., 2009; DeNero and Uszkor- eit, 2011; Neubig et al., 2012). We are inter- ested in practical, language-independent preorder- ing approaches that rely only on automatic source- language parsers (Genzel, 2010). The most recent work in this area uses large-scale feature-rich dis- riminative models, effectively treating preordering either as a learning to rank (Yang et al., 2012), multi- classification (Lerner and Petrov, 2013) or logistic regression (Jehl et al., 2014) problem. In this paper we incorporate NNs into the latter approach. Lately an increasing body …

Content truncated to 500 words