Drem: The AFRL Submission to the WMT15 Tuning Task

Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 422–427, Lisboa, Portugal, 17-18 September 2015. c⃝2015 Association for Computational Linguistics. Drem: The AFRL Submission to the WMT15 Tuning Task Grant Erdmann Air Force Research Laboratory grant.erdmann@us.af.mil Jeremy Gwinnup SRA International† jeremy.gwinnup.ctr@us.af.mil Abstract We define a new algorithm, named “Drem”, for tuning the weighted linear model in a statistical machine translation system. Drem has two major innova- tions. First, it uses scaled derivative-free trust-region optimization rather than other methods’ line search or (sub)gradient ap- proximations. Second, it interpolates the decoder output, using information about which decodes produced which transla- tions. 1 Introduction While searching for the best translation of a text, statistical machine translation systems generate several different quantitative descriptors of the translation, called “features”. These features are combined into a single score, by weighting and summing them. A tuning algorithm chooses the weights used in this combination. MERT (Och, 2003) is the standard tuning al- gorithm. Many different varieties of error rate training exist, with various techniques, includ- ing expectation, line-search, Nelder–Mead sim- plex (Zhao and Chen, 2009), particle swarm op- timization (Suzuki et al., 2011), and stabilization (Foster and Kuhn, 2009). It has been experienced that MERT fails to perform well in larger feature spaces, but recently there has been evidence of a regularized MERT succeeding in high dimensions (Galley et al., 2013). Other methods have been designed as wholesale replacements for MERT, including MIRA (Chiang et al., 2008), k-best MIRA (Cherry and Foster, 2012), PRO (Hopkins and May, 2011), and Ram- pion (Gimpel and Smith, 2012). †This work is sponsored by the Air Force Research Labo- ratory under Air Force contract FA-8650-09-D-6939-029. MERT’s continued use indicates an improved dense-feature optimizer for weighted linear mod- els would be welcome. It is in this context that we introduce our tuning algorithm, named “Drem”. In contrast to known varieties of MERT, Drem is not a line-search, simplex, or particle swarm optimiza- tion method. It is a derivative-free trust-region method, with several advancements to cater to the particular nature of MT system optimization. 2 Background and definitions A feature vector f ∈ Rk has the relative impor- tance of its components determined by a weight vector w ∈ Rk. For a weighted linear model, the score used by the decoder to choose the best trans- lation is the scalar product, s(w, f) = wT f (1) which we call the decoder score. The output of the decoder run on a corpus C at a weight w is an n- best list N(w, C). The n-best list can be thought of as a collection of elements of the form (j, t, f), where j ∈ J is the segment (typically sentence) index within C, t is the text of the translation, and f is the feature vector. The objective of tuning is to choose the weights w such that the translation with the highest decoder score (i.e., the “1-best”) will be the segment’s best translation. For the test error a human performs …

Content truncated to 500 words