A Systematic Comparison of Training Criteria for Statistical Machine Translation

Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 524–532, Prague, June 2007. c⃝2007 Association for Computational Linguistics A Systematic Comparison of Training Criteria for Statistical Machine Translation Richard Zens and Saˇsa Hasan and Hermann Ney Human Language Technology and Pattern Recognition Lehrstuhl f¨ur Informatik 6 – Computer Science Department RWTH Aachen University, D-52056 Aachen, Germany {zens,hasan,ney}@cs.rwth-aachen.de Abstract We address the problem of training the free parameters of a statistical machine transla- tion system. We show significant improve- ments over a state-of-the-art minimum er- ror rate training baseline on a large Chinese- English translation task. We present novel training criteria based on maximum likeli- hood estimation and expected loss compu- tation. Additionally, we compare the maxi- mum a-posteriori decision rule and the min- imum Bayes risk decision rule. We show that, not only from a theoretical point of view but also in terms of translation qual- ity, the minimum Bayes risk decision rule is preferable. 1 Introduction Once we specified the Bayes decision rule for statis- tical machine translation, we have to address three problems (Ney, 2001): • the search problem, i.e. how to find the best translation candidate among all possible target language sentences; • the modeling problem, i.e. how to structure the dependencies of source and target language sentences; • the training problem, i.e. how to estimate the free parameters of the models from the training data. Here, the main focus is on the training problem. We will compare a variety of training criteria for statisti- cal machine translation. In particular, we are consid- ering criteria for the log-linear parameters or model scaling factors. We will introduce new training cri- teria based on maximum likelihood estimation and expected loss computation. We will show that some achieve significantly better results than the standard minimum error rate training of (Och, 2003). Additionally, we will compare two decision rules, the common maximum a-posteriori (MAP) deci- sion rule and the minimum Bayes risk (MBR) de- cision rule (Kumar and Byrne, 2004). We will show that the minimum Bayes risk decision rule results in better translation quality than the maximum a- posteriori decision rule for several training criteria. The remaining part of this paper is structured as follows: first, we will describe related work in Sec. 2. Then, we will briefly review the baseline system, Bayes decision rule for statistical machine translation and automatic evaluation metrics for ma- chine translation in Sec. 3 and Sec. 4, respectively. The novel training criteria are described in Sec. 5 and Sec. 6. Experimental results are reported in Sec. 7 and conclusions are given in Sec. 8. 2 Related Work The most common modeling approach in statistical machine translation is to use a log-linear combina- tion of several sub-models (Och and Ney, 2002). In (Och and Ney, 2002), the log-linear weights were tuned to maximize the mutual information criterion (MMI). The current state-of-the-art is to optimize these parameters with respect to the final evaluation criterion; this is …

Content truncated to 500 words