Model Combination for Machine Translation

Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL, pages 975–983, Los Angeles, California, June 2010. c⃝2010 Association for Computational Linguistics Model Combination for Machine Translation John DeNero, Shankar Kumar, Ciprian Chelba, and Franz Och UC Berkeley Google, Inc. denero@berkeley.edu {shankarkumar,ciprianchelba,och}@google.com Abstract Machine translation benefits from two types of decoding techniques: consensus decoding over multiple hypotheses under a single model and system combination over hypotheses from different models. We present model combina- tion, a method that integrates consensus de- coding and system combination into a uni- fied, forest-based technique. Our approach makes few assumptions about the underly- ing component models, enabling us to com- bine systems with heterogenous structure. Un- like most system combination techniques, we reuse the search space of component models, which entirely avoids the need to align trans- lation hypotheses. Despite its relative sim- plicity, model combination improves trans- lation quality over a pipelined approach of first applying consensus decoding to individ- ual systems, and then applying system combi- nation to their output. We demonstrate BLEU improvements across data sets and language pairs in large-scale experiments. 1 Introduction Once statistical translation models are trained, a de- coding approach determines what translations are fi- nally selected. Two parallel lines of research have shown consistent improvements over the standard max-derivation decoding objective, which selects the highest probability derivation. Consensus de- coding procedures select translations for a single system by optimizing for model predictions about n-grams, motivated either as minimizing Bayes risk (Kumar and Byrne, 2004), maximizing sentence similarity (DeNero et al., 2009), or approximating a max-translation objective (Li et al., 2009b). System combination procedures, on the other hand, generate translations from the output of multiple component systems (Frederking and Nirenburg, 1994). In this paper, we present model combination, a technique that unifies these two approaches by learning a con- sensus model over the n-gram features of multiple underlying component models. Model combination operates over the compo- nent models’ posterior distributions over translation derivations, encoded as a forest of derivations.1 We combine these components by constructing a linear consensus model that includes features from each component. We then optimize this consensus model over the space of all translation derivations in the support of all component models’ posterior distribu- tions. By reusing the components’ search spaces, we entirely avoid the hypothesis alignment problem that is central to standard system combination ap- proaches (Rosti et al., 2007). Forest-based consensus decoding techniques dif- fer in whether they capture model predictions through n-gram posteriors (Tromble et al., 2008; Kumar et al., 2009) or expected n-gram counts (DeNero et al., 2009; Li et al., 2009b). We evaluate both in controlled experiments, demonstrating their empirical similarity. We also describe algorithms for expanding translation forests to ensure that n-grams are local to a forest’s hyperedges, and for exactly computing n-gram posteriors efficiently. Model combination assumes only that each trans- lation model can produce expectations of n-gram features; the latent derivation structures of compo- nent systems can differ arbitrarily. This flexibility allows …

Content truncated to 500 words