Minimum Bayes-Risk Decoding for Statistical Machine Translation

Minimum Bayes-Risk Decoding for Statistical Machine Translation Shankar Kumar and William Byrne � Center for Language and Speech Processing, Johns Hopkins University, 3400 North Charles Street, Baltimore, MD, 21218, USA � skumar,byrne � @jhu.edu Abstract We present Minimum Bayes-Risk (MBR) de- coding for statistical machine translation. This statistical approach aims to minimize expected loss of translation errors under loss functions that measure translation performance. We de- scribe a hierarchy of loss functions that incor- porate different levels of linguistic information from word strings, word-to-word alignments from an MT system, and syntactic structure from parse-trees of source and target language sentences. We report the performance of the MBR decoders on a Chinese-to-English trans- lation task. Our results show that MBR decod- ing can be used to tune statistical MT perfor- mance for specific loss functions. 1 Introduction Statistical Machine Translation systems have achieved considerable progress in recent years as seen from their performance on international competitions in standard evaluation tasks (NIST, 2003). This rapid progress has been greatly facilitated by the development of automatic translation evaluation metrics such as BLEU score (Pa- pineni et al., 2001), NIST score (Doddington, 2002) and Position Independent Word Error Rate (PER) (Och, 2002). However, given the many factors that influence translation quality, it is unlikely that we will find a single translation metric that will be able to judge all these fac- tors. For example, the BLEU, NIST and the PER metrics, � This work was supported by the National Science Foun- dation under Grant No. 0121285 and an ONR MURI Grant N00014-01-1-0685. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the au- thors and do not necessarily reflect the views of the National Science Foundation or the Office of Naval Research. though effective, do not take into account explicit syntac- tic information when measuring translation quality. Given that different Machine Translation (MT) eval- uation metrics are useful for capturing different aspects of translation quality, it becomes desirable to create MT systems tuned with respect to each individual criterion. In contrast, the maximum likelihood techniques that under- lie the decision processes of most current MT systems do not take into account these application specific goals. We apply the Minimum Bayes-Risk (MBR) techniques devel- oped for automatic speech recognition (Goel and Byrne, 2000) and bitext word alignment for statistical MT (Ku- mar and Byrne, 2002), to the problem of building au- tomatic MT systems tuned for specific metrics. This is a framework that can be used with statistical models of speech and language to develop decision processes opti- mized for specific loss functions. We will show that MBR decoding can be applied to machine translation in two scenarios. Given an automatic MT metric, we design a loss function based on the met- ric and use MBR decoding to tune MT performance un- der the metric. We also show how MBR decoding can be used to incorporate syntactic structure into a statistical MT system by building specialized loss functions. These loss …

Content truncated to 500 words