Review of Hypothesis Alignment Algorithms for MT System Combination via Confusion Network Decoding

Proceedings of the 7th Workshop on Statistical Machine Translation, pages 191–199, Montr´eal, Canada, June 7-8, 2012. c⃝2012 Association for Computational Linguistics Review of Hypothesis Alignment Algorithms for MT System Combination via Confusion Network Decoding Antti-Veikko I. Rostia∗, Xiaodong Heb, Damianos Karakosc, Gregor Leuschd†, Yuan Caoc, Markus Freitage, Spyros Matsoukasf, Hermann Neye, Jason R. Smithc and Bing Zhangf aApple Inc., Cupertino, CA 95014 arosti@apple.com bMicrosoft Research, Redmond, WA 98052 xiaohe@microsoft.com cJohns Hopkins University, Baltimore, MD 21218 {damianos,yuan.cao,jrsmith}@jhu.edu dSAIC, Monheimsallee 22, D-52062 Aachen, Germany gregor.leusch@saic.com eRWTH Aachen University, D-52056 Aachen, Germany {freitag,ney}@cs.rwth-aachen.de fRaytheon BBN Technologies, 10 Moulton Street, Cambridge, MA 02138 {smatsouk,bzhang}@bbn.com Abstract Confusion network decoding has proven to be one of the most successful approaches to machine translation system combination. The hypothesis alignment algorithm is a cru- cial part of building the confusion networks and many alternatives have been proposed in the literature. This paper describes a sys- tematic comparison of five well known hy- pothesis alignment algorithms for MT sys- tem combination via confusion network de- coding. Controlled experiments using identi- cal pre-processing, decoding, and weight tun- ing methods on standard system combina- tion evaluation sets are presented. Transla- tion quality is assessed using case insensitive BLEU scores and bootstrapping is used to es- tablish statistical significance of the score dif- ferences. All aligners yield significant BLEU score gains over the best individual system in- cluded in the combination. Incremental indi- rect hidden Markov model and a novel incre- mental inversion transduction grammar with flexible matching consistently yield the best translation quality, though keeping all things equal, the differences between aligners are rel- atively small. ∗The work reported in this paper was carried out while the authors were at Raytheon BBN Technologies and †RWTH Aachen University. 1 Introduction Current machine translation (MT) systems are based on different paradigms, such as rule-based, phrase- based, hierarchical, and syntax-based. Due to the complexity of the problem, systems make various assumptions at different levels of processing and modeling. Many of these assumptions may be suboptimal and complementary. The complemen- tary information in the outputs from multiple MT systems may be exploited by system combination. Availability of multiple system outputs within the DARPA GALE program as well as NIST Open MT and Workshop on Statistical Machine Translation evaluations has led to extensive research in combin- ing the strengths of diverse MT systems, resulting in significant gains in translation quality. System combination methods proposed in the lit- erature can be roughly divided into three categories: (i) hypothesis selection (Rosti et al., 2007b; Hilde- brand and Vogel, 2008), (ii) re-decoding (Frederking and Nirenburg, 1994; Jayaraman and Lavie, 2005; Rosti et al., 2007b; He and Toutanova, 2009; De- vlin et al., 2011), and (iii) confusion network de- coding. Confusion network decoding has proven to be the most popular as it does not require deep N- best lists1 and operates on the surface strings. It has 1N-best lists of around N = 10 have been used in confu- sion network decoding yielding small gains over using 1-best 191 also …

Content truncated to 500 words