Using Paraphrases for Parameter Tuning in Statistical Machine Translation

Proceedings of the Second Workshop on Statistical Machine Translation, pages 120–127, Prague, June 2007. c⃝2007 Association for Computational Linguistics Using Paraphrases for Parameter Tuning in Statistical Machine Translation Nitin Madnani, Necip Fazil Ayan, Philip Resnik & Bonnie J. Dorr Institute for Advanced Computer Studies University of Maryland College Park, MD, 20742 {nmadnani,nfa,resnik,bonnie}@umiacs.umd.edu Abstract Most state-of-the-art statistical machine translation systems use log-linear models, which are defined in terms of hypothesis fea- tures and weights for those features. It is standard to tune the feature weights in or- der to maximize a translation quality met- ric, using held-out test sentences and their corresponding reference translations. How- ever, obtaining reference translations is ex- pensive. In this paper, we introduce a new full-sentence paraphrase technique, based on English-to-English decoding with an MT system, and we demonstrate that the result- ing paraphrases can be used to drastically re- duce the number of human reference transla- tions needed for parameter tuning, without a significant decrease in translation quality. 1 Introduction Viewed at a very high level, statistical machine translation involves four phases: language and trans- lation model training, parameter tuning, decoding, and evaluation (Lopez, 2007; Koehn et al., 2003). Since their introduction in statistical MT by Och and Ney (2002), log-linear models have been a standard way to combine sub-models in MT systems. Typi- cally such a model takes the form � i λiφi( ¯f, ¯e) (1) where φi are features of the hypothesis e and λi are weights associated with those features. Selecting appropriate weights λi is essential in order to obtain good translation performance. Och (2003) introduced minimum error rate train- ing (MERT), a technique for optimizing log-linear model parameters relative to a measure of translation quality. This has become much more standard than optimizing the conditional probability of the train- ing data given the model (i.e., a maximum likelihood criterion), as was common previously. Och showed that system performance is best when parameters are optimized using the same objective function that will be used for evaluation; BLEU (Papineni et al., 2002) remains common for both purposes and is often re- tained for parameter optimization even when alter- native evaluation measures are used, e.g., (Banerjee and Lavie, 2005; Snover et al., 2006). Minimum error rate training—and more gener- ally, optimization of parameters relative to a trans- lation quality measure—relies on data sets in which source language sentences are paired with (sets of) reference translations. It is widely agreed that, at least for the widely used BLEU criterion, which is based on n-gram overlap between hypotheses and reference translations, the criterion is most accu- rate when computed with as many distinct reference translations as possible. Intuitively this makes sense: if there are alternative ways to phrase the meaning of the source sentence in the target language, then the translation quality criterion should take as many of those variations into account as possible. To do otherwise is to risk the possibility that the criterion might judge good translations to be poor when they fail to match …

Content truncated to 500 words