Lattice Minimum Bayes-Risk Decoding for Statistical Machine Translation

Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 620–629, Honolulu, October 2008. c⃝2008 Association for Computational Linguistics Lattice Minimum Bayes-Risk Decoding for Statistical Machine Translation Roy W. Tromble1 and Shankar Kumar2 and Franz Och2 and Wolfgang Macherey2 1Department of Computer Science Johns Hopkins University Baltimore, MD 21218, USA royt@jhu.edu 2 Google Inc. 1600 Amphitheatre Pkwy. Mountain View, CA 94043, USA {shankarkumar,och,wmach}@google.com Abstract We present Minimum Bayes-Risk (MBR) de- coding over translation lattices that compactly encode a huge number of translation hypothe- ses. We describe conditions on the loss func- tion that will enable efficient implementation of MBR decoders on lattices. We introduce an approximation to the BLEU score (Pap- ineni et al., 2001) that satisfies these condi- tions. The MBR decoding under this approx- imate BLEU is realized using Weighted Fi- nite State Automata. Our experiments show that the Lattice MBR decoder yields mod- erate, consistent gains in translation perfor- mance over N-best MBR decoding on Arabic- to-English, Chinese-to-English and English- to-Chinese translation tasks. We conduct a range of experiments to understand why Lat- tice MBR improves upon N-best MBR and study the impact of various parameters on MBR performance. 1 Introduction Statistical language processing systems for speech recognition, machine translation or parsing typically employ the Maximum A Posteriori (MAP) deci- sion rule which optimizes the 0-1 loss function. In contrast, these systems are evaluated using metrics based on string-edit distance (Word Error Rate), n- gram overlap (BLEU score (Papineni et al., 2001)), or precision/recall relative to human annotations. Minimum Bayes-Risk (MBR) decoding (Bickel and Doksum, 1977) aims to address this mismatch by se- lecting the hypothesis that minimizes the expected error in classification. Thus it directly incorporates the loss function into the decision criterion. The ap- proach has been shown to give improvements over the MAP classifier in many areas of natural lan- guage processing including automatic speech recog- nition (Goel and Byrne, 2000), machine transla- tion (Kumar and Byrne, 2004; Zhang and Gildea, 2008), bilingual word alignment (Kumar and Byrne, 2002), and parsing (Goodman, 1996; Titov and Hen- derson, 2006; Smith and Smith, 2007). In statistical machine translation, MBR decoding is generally implemented by re-ranking an N-best list of translations produced by a first-pass decoder; this list typically contains between 100 and 10, 000 hypotheses. Kumar and Byrne (2004) show that MBR decoding gives optimal performance when the loss function is matched to the evaluation criterion; in particular, MBR under the sentence-level BLEU loss function (Papineni et al., 2001) gives gains on BLEU. This is despite the fact that the sentence-level BLEU loss function is an approximation to the exact corpus-level BLEU. A different MBR inspired decoding approach is pursued in Zhang and Gildea (2008) for machine translation using Synchronous Context Free Gram- mars. A forest generated by an initial decoding pass is rescored using dynamic programming to maxi- mize the expected count of synchronous constituents in the tree that corresponds to the translation. Since each constituent adds a new 4-gram to the …

Content truncated to 500 words