Towards Efficient Large-Scale Feature-Rich Statistical Machine Translation

Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 128–133, Sofia, Bulgaria, August 8-9, 2013 c⃝2013 Association for Computational Linguistics Towards Efficient Large-Scale Feature-Rich Statistical Machine Translation Vladimir Eidelman1, Ke Wu1, Ferhan Ture1, Philip Resnik2, Jimmy Lin3 1 Dept. of Computer Science 2 Dept. of Linguistics 3 The iSchool Institute for Advanced Computer Studies University of Maryland {vlad,wuke,fture,resnik,jimmylin}@umiacs.umd.edu Abstract We present the system we developed to provide efficient large-scale feature-rich discriminative training for machine trans- lation. We describe how we integrate with MapReduce using Hadoop streaming to allow arbitrarily scaling the tuning set and utilizing a sparse feature set. We report our findings on German-English and Russian- English translation, and discuss benefits, as well as obstacles, to tuning on larger development sets drawn from the parallel training data. 1 Introduction The adoption of discriminative learning methods for SMT that scale easily to handle sparse and lex- icalized features has been increasing in the last several years (Chiang, 2012; Hopkins and May, 2011). However, relatively few systems take full advantage of the opportunity. With some excep- tions (Simianer et al., 2012), most still rely on tuning a handful of common dense features, along with at most a few thousand others, on a relatively small development set (Cherry and Foster, 2012; Chiang et al., 2009). While more features tuned on more data usually results in better performance for other NLP tasks, this has not necessarily been the case for SMT. Thus, our main focus in this paper is to improve understanding into the effective use of sparse fea- tures, and understand the benefits and shortcom- ings of large-scale discriminative training. To this end, we conducted experiments for the shared translation task of the 2013 Workshop on Statis- tical Machine Translation for the German-English and Russian-English language pairs. 2 Baseline system We use a hierarchical phrase-based decoder im- plemented in the open source translation system cdec1 (Dyer et al., 2010). For tuning, we use Mr. MIRA2 (Eidelman et al., 2013), an open source decoder agnostic implementation of online large-margin learning in Hadoop MapReduce. Mr. MIRA separates learning from the decoder, allow- ing the flexibility to specify the desired inference procedure through a simple text communication protocol. The decoder receives input sentences and weight updates from the learner, while the learner receives k-best output with feature vectors from the decoder. Hadoop MapReduce (Dean and Ghemawat, 2004) is a popular distributed processing frame- work that has gained widespread adoption, with the advantage of providing scalable parallelization in a manageable framework, taking care of data distribution, synchronization, fault tolerance, as well as other features. Thus, while we could oth- erwise achieve the same level of parallelization, it would be in a more ad-hoc manner. The advantage of online methods lies in their ability to deal with large training sets and high- dimensional input representations while remain- ing simple and offering fast convergence. With Hadoop streaming, our system can take advantage of commodity clusters to handle parallel large- scale training while also being capable of running …

Content truncated to 500 words