Large-scale Reordering Model for Statistical Machine Translation using Dual Multinomial Logistic Regression

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1758–1763, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Large-scale Reordering Model for Statistical Machine Translation using Dual Multinomial Logistic Regression Abdullah Alrajehab and Mahesan Niranjanb aComputer Research Institute, King Abdulaziz City for Science and Technology (KACST) Riyadh, Saudi Arabia, asrajeh@kacst.edu.sa bSchool of Electronics and Computer Science, University of Southampton Southampton, United Kingdom, {asar1a10, mn}@ecs.soton.ac.uk Abstract Phrase reordering is a challenge for statis- tical machine translation systems. Posing phrase movements as a prediction prob- lem using contextual features modeled by maximum entropy-based classifier is su- perior to the commonly used lexicalized reordering model. However, Training this discriminative model using large-scale parallel corpus might be computationally expensive. In this paper, we explore recent advancements in solving large-scale clas- sification problems. Using the dual prob- lem to multinomial logistic regression, we managed to shrink the training data while iterating and produce significant saving in computation and memory while preserv- ing the accuracy. 1 Introduction Phrase reordering is a common problem when translating between two grammatically different languages. Analogous to speech recognition sys- tems, statistical machine translation (SMT) sys- tems relied on language models to produce more fluent output. While early work penalized phrase movements without considering reorderings aris- ing from vastly differing grammatical structures across language pairs like Arabic-English (Koehn, 2004a), many researchers considered lexicalized reordering models that attempted to learn orienta- tion based on the training corpus (Tillmann, 2004; Kumar and Byrne, 2005; Koehn et al., 2005). Building on this, some researchers have bor- rowed powerful ideas from the machine learning literature, to pose the phrase movement problem as a prediction problem using contextual input fea- tures whose importance is modeled as weights of a linear classifier trained by entropic criteria. The approach (so called maximum entropy classifier or simply MaxEnt) is a popular choice (Zens and Ney, 2006; Xiong et al., 2006; Nguyen et al., 2009; Xiang et al., 2011). Max-margin structure classifiers were also proposed (Ni et al., 2011). Alternatively, Cherry (2013) proposed recently us- ing sparse features optimize the translation quality with the decoder instead of training a classifier in- dependently. While large-scale parallel corpus is advanta- geous for improving such reordering model, this improvement comes at a price of computational complexity. This issue is particularly pronounced when discriminative models are considered such as maximum entropy-based model due to the re- quired iterative learning. Advancements in solving large-scale classifica- tion problems have been shown to be effective such as dual coordinate descent method for linear support vector machines (Hsieh et al., 2008). Sim- ilarly, Yu et al. (2011) proposed a two-level dual coordinate descent method for maximum entropy classifier. In this work we explore the dual problem to multinomial logistic regression for building large- scale reordering model (section 3). One of the main advantages of solving the dual problem is providing a mechanism to shrink the training data which is a serious issue in building such large- scale system. We present empirical …

Content truncated to 500 words