Robust Estimation of Feature Weights in Statistical Machine Translation

Robust Estimation of Feature Weights in Statistical Machine Translation Cristina Espa˜na-Bonet and Llu´ıs M`arquez TALP Research Center Universitat Polit`ecnica de Catalunya Jordi Girona 1–3, 08034 Barcelona cristinae@lsi.upc.edu, lluism@lsi.upc.edu Abstract Weights of the various components in a standard Statistical Machine Translation model are usually estimated via Minimum Error Rate Training. With this, one finds their optimum value on a development set with the expectation that these opti- mal weights generalise well to other test sets. However, this is not always the case when domains differ. This work uses a perceptron algorithm to learn more ro- bust weights to be used on out-of-domain corpora without the need for specialised data. For an Arabic-to-English translation system, the generalisation of weights rep- resents an improvement of more than 2 points of BLEU with respect to the MERT baseline using the same information. 1 Introduction In Statistical Machine Translation (SMT) and within the log-linear model (Och and Ney, 2002), the best translation ˆe for a given source sentence f is the most probable one, and the probability is expressed as a weighted sum of different elements: T(f) = ˆe = argmaxe � m λmhm(f, e) . (1) In the standard most simple form, one consid- ers 8 components being hm(f, e) log-probabilities of: the language model P(e), the generative and discriminative lexical translation probabilities lex(f|e) and lex(e|f) respectively, the genera- tive and discriminative translation models P(f|e) and P(e|f), the distortion model Pd(e, f), and the phrase and word penalties, ph(e) and w(e). c⃝ 2010 European Association for Machine Translation. The λ weights, which account for the relative importance of each feature in the log-linear prob- abilistic model, are commonly estimated by opti- mising the translation performance on a develop- ment set. For this optimisation one can use Min- imum Error Rate Training (MERT) (Och, 2003) where BLEU (Papineni et al., 2002) is the refer- ence score. MERT estimates the 8D best fit by searching the minimum in each dimension of the parame- ter space. The line search used in Och (2003) is demonstrated to find the absolute minimum in that direction, still, this does not guarantee that the best parameters obtained are the optimum ones. In fact, the larger the number of features, the less reliable the global minimisation will be. Some works such as Cer et al. (2008), Moore and Quirk (2008), or Foster and Kuhn (2009) try to improve the stan- dard MERT minimisation. Here we do not follow this line, since we are not interested in finding the optimal parameters on development but on test. In this study, we see how parameters estimated with MERT can generalise quite bad on test sets that depart substantially from the training and de- velopment sets. Our goal is to find a more robust vector of weights −→λ that, even without being op- timal on development or test when the domain is akin, better generalise on the other cases. We show empirically that this can be done by including ma- chine learning techiques to estimate −→λ …

Content truncated to 500 words