Bayesian Reordering Model with Feature Selection

Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 477–485, Baltimore, Maryland USA, June 26–27, 2014. c⃝2014 Association for Computational Linguistics Bayesian Reordering Model with Feature Selection Abdullah Alrajehab and Mahesan Niranjanb aComputer Research Institute, King Abdulaziz City for Science and Technology (KACST) Riyadh, Saudi Arabia, asrajeh@kacst.edu.sa bSchool of Electronics and Computer Science, University of Southampton Southampton, United Kingdom, {asar1a10, mn}@ecs.soton.ac.uk Abstract In phrase-based statistical machine trans- lation systems, variation in grammatical structures between source and target lan- guages can cause large movements of phrases. Modeling such movements is cru- cial in achieving translations of long sen- tences that appear natural in the target lan- guage. We explore generative learning approach to phrase reordering in Arabic to English. Formulating the reordering problem as a classification problem and using naive Bayes with feature selection, we achieve an improvement in the BLEU score over a lexicalized reordering model. The proposed model is compact, fast and scalable to a large corpus. 1 Introduction Currently, the dominant approach to machine translation is statistical, starting from the math- ematical formulations and algorithms for param- eter estimation (Brown et al., 1988), further ex- tended in (Brown et al., 1993). These early mod- els, widely known as the IBM models, were word- based. Recent extensions note that a better ap- proach is to group collections of words, or phrases, for translation together, resulting in a significant focus these days on phrase-based statistical ma- chine translation systems. To deal with the alignment problem of one- to-many word alignments in the IBM model formulation, whereas phrase-based models may have many-to-many translation relationships, IBM models are trained in both directions, source to tar- get and target to source, and their word alignments are combined (Och and Ney, 2004). While phrase-based systems are a significant improvement over word-based approaches, a par- ticular issue that emerges is long-range reorder- ings at the phrase level (Galley and Manning, 2008). Analogous to speech recognition systems, translation systems relied on language models to produce more fluent translation. While early work penalized phrase movements without considering reorderings arising from vastly differing grammat- ical structures across language pairs like Arabic- English, many researchers considered lexical re- ordering models that attempted to learn orienta- tion based on content (Tillmann, 2004; Kumar and Byrne, 2005; Koehn et al., 2005). These approaches may suffer from the data sparseness problem since many phrase pairs occur only once (Nguyen et al., 2009). As an alternative way of exploiting function ap- proximation capabilities offered by machine learn- ing methods, there is recent interest in formulating a learning problem that aims to predict reorder- ing from linguistic features that capture their con- text. An example of this is the maximum entropy method used by (Xiang et al., 2011; Nguyen et al., 2009; Zens and Ney, 2006; Xiong et al., 2006). In this work we apply a naive Bayes classifier, combined with feature selection to address the re- ordering problem. To the best of our knowledge, this simple model of classification has not been …

Content truncated to 500 words