Combining Outputs from Multiple Machine Translation Systems

Proceedings of NAACL HLT 2007, pages 228–235, Rochester, NY, April 2007. c⃝2007 Association for Computational Linguistics Combining Outputs from Multiple Machine Translation Systems Antti-Veikko I. Rosti � and Necip Fazil Ayan � and Bing Xiang � and Spyros Matsoukas � and Richard Schwartz � and Bonnie J. Dorr � � BBN Technologies, 10 Moulton Street, Cambridge, MA 02138 � arosti,bxiang,smatsouk,schwartz � @bbn.com � Institute for Advanced Computer Studies, University of Maryland, College Park, MD 20742 � nfa,bonnie � @umiacs.umd.edu Abstract Currently there are several approaches to machine translation (MT) based on differ- ent paradigms; e.g., phrasal, hierarchical and syntax-based. These three approaches yield similar translation accuracy despite using fairly different levels of linguistic knowledge. The availability of such a variety of systems has led to a growing interest toward finding better translations by combining outputs from multiple sys- tems. This paper describes three differ- ent approaches to MT system combina- tion. These combination methods oper- ate on sentence, phrase and word level exploiting information from � -best lists, system scores and target-to-source phrase alignments. The word-level combination provides the most robust gains but the best results on the development test sets (NIST MT05 and the newsgroup portion of GALE 2006 dry-run) were achieved by combining all three methods. 1 Introduction In recent years, machine translation systems based on new paradigms have emerged. These systems employ more than just the surface-level information used by the state-of-the-art phrase-based translation systems. For example, hierarchical (Chiang, 2005) and syntax-based (Galley et al., 2006) systems have recently improved in both accuracy and scalability. Combined with the latest advances in phrase-based translation systems, it has become more attractive to take advantage of the various outputs in forming consensus translations (Frederking and Nirenburg, 1994; Bangalore et al., 2001; Jayaraman and Lavie, 2005; Matusov et al., 2006). System combination has been successfully ap- plied in state-of-the-art speech recognition evalua- tion systems for several years (Fiscus, 1997). Even though the underlying modeling techniques are sim- ilar, many systems produce very different outputs with approximately the same accuracy. One of the most successful approaches is consensus network decoding (Mangu et al., 2000) which assumes that the confidence of a word in a certain position is based on the sum of confidences from each system output having the word in that position. This re- quires aligning the system outputs to form a con- sensus network and – during decoding – simply finding the highest scoring path through this net- work. The alignment of speech recognition outputs is fairly straightforward due to the strict constraint in word order. However, machine translation outputs do not have this constraint as the word order may be different between the source and target languages. MT systems employ various re-ordering (distortion) models to take this into account. Three MT system combination methods are pre- sented in this paper. They operate on the sentence, phrase and word level. The sentence-level combi- nation is based on selecting the best hypothesis out of the merged N-best lists. This method does …

Content truncated to 500 words