Positive Diversity Tuning for Machine Translation System Combination

Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 320–328, Sofia, Bulgaria, August 8-9, 2013 c⃝2013 Association for Computational Linguistics Positive Diversity Tuning for Machine Translation System Combination Daniel Cer, Christopher D. Manning and Daniel Jurafsky Stanford University Stanford, CA 94305, USA {danielcer,manning,jurafsky}@stanford.edu Abstract We present Positive Diversity Tuning, a new method for tuning machine translation models specifically for improved perfor- mance during system combination. Sys- tem combination gains are often limited by the fact that the translations produced by the different component systems are too similar to each other. We propose a method for reducing excess cross-system similarity by optimizing a joint objective that simultaneously rewards models for producing translations that are similar to reference translations, while also punish- ing them for translations that are too sim- ilar to those produced by other systems. The formulation of the Positive Diversity objective is easy to implement and allows for its quick integration with most machine translation tuning pipelines. We find that individual systems tuned on the same data to Positive Diversity can be even more diverse than systems built using different data sets, while still obtaining good BLEU scores. When these individual systems are used together for system combination, our approach allows for significant gains of 0.8 BLEU even when the combination is per- formed using a small number of otherwise identical individual systems. 1 Introduction The best performing machine translation sys- tems are typically not individual decoders but rather are ensembles of two or more systems whose output is then merged using system combination algorithms. Since combining multiple distinct equally good translation systems reliably produces gains over any one of the systems in isolation, it is widely used in situations where high quality is es- sential. Exploiting system combination brings signifi- cant cost: Macherey and Och (2007) showed that successful system combination requires the con- struction of multiple systems that are simultane- ously diverse and well-performing. If the systems are not distinct enough, they will bring very lit- tle value during system combination. However, if some of the systems produce diverse transla- tions but achieve lower overall translation quality, their contributions risk being ignored during sys- tem combination. Prior work has approached the need for diverse systems by using different system architectures, model components, system build parameters, de- coder hyperparameters, as well as data selection and weighting (Macherey and Och, 2007; DeNero et al., 2010; Xiao et al., 2013). However, during tuning, each individual system is still just trained to maximize its own isolated performance on a tune set, or at best an error-driven reweighting of the tune set, without explicitly taking into account the diversity of the resulting translations. Such tuning does not encourage systems to rigorously explore model variations that achieve both good translation quality and diversity with respect to the other sys- tems. It is reasonable to suspect that this results in individual systems that under exploit the amount of diversity possible, given the characteristics of the individual systems. For better system combination, we propose …

Content truncated to 500 words