CMU System Combination in WMT 2011

Proceedings of the 6th Workshop on Statistical Machine Translation, pages 145–151, Edinburgh, Scotland, UK, July 30–31, 2011. c⃝2011 Association for Computational Linguistics CMU System Combination in WMT 2011 Kenneth Heafield and Alon Lavie Carnegie Mellon University 5000 Forbes Ave Pittsburgh, PA, USA {heafield,alavie}@cs.cmu.edu Abstract This paper describes our submissions, cmu-heafield-combo, to the ten tracks of the 2011 Workshop on Machine Transla- tion’s system combination task. We show how the combination scheme operates by flexibly aligning system outputs then searching a space constructed from the alignments. Humans judged our combination the best on eight of ten tracks. 1 Introduction We participated in all ten tracks of the 2011 Work- shop on Machine Translation system combination task as cmu-heafield-combo. This uses a sys- tem combination scheme that builds on our prior work (Heafield and Lavie, 2010), especially with respect to language modeling and handling non- English languages. We present a summary of the system, describe improvements, list the data used (all of the constrained monolingual data), and present automatic results in anticipation of human evaluation by the workshop. 2 Our Combination Scheme Given single-best outputs from each system, the scheme aligns system outputs then searches a space based on these alignments. The scheme is a contin- uation of our previous system (Heafield and Lavie, 2010) so we describe unchanged parts of the sys- tem in less detail, preferring instead to focus on new components. 2.1 Alignment We run the METEOR matcher (Denkowski and Lavie, 2010) on every pair of system outputs for a given sentence. It identifies exact matches, identi- cal stems (Porter, 2001) except for Czech, WordNet synonym matches for English (Fellbaum, 1998), and automatically extracted matches for all five target languages. The automatic matches come from piv- oting (Bannard and Callison-Burch, 2005) on con- strained data. An example METEOR alignment is shown in Figure 1, though it need not be monotone. Twice that produced by nuclear plants Double that that produce nuclear power stations Figure 1: Alignment generated by METEOR showing exact (that–that and nuclear–nuclear), stem (produced– produce), synonym (twice–double), and unigram para- phrase (plants–stations) alignments. 2.2 Search The search space is unchanged from Heafield and Lavie (2010), so we give a summary here. The gen- eral idea is to generate a combined sentence one word at a time, going from left to right. As the scheme creates an output, it also steps through the system outputs from left to right. Stepping through systems is synchronized with the partial output, so that words to the left are already captured in the hy- pothesis and the next word from any of the systems represents a meaningful extension of the partial out- put. All of these options are considered by hypothe- sis branching. 145 Thus far, we have assumed that system outputs are monotone: they agree on word order, so it is possi- ble to step through all of them simultaneously. On the left are words captured in the partial output and on the right are the words whose meaning remains to …

Content truncated to 500 words