An Augmented Three-Pass System Combination Framework: DCU Combination System for WMT 2010

Proceedings of the Joint 5th Workshop on Statistical Machine Translation and MetricsMATR, pages 290–295, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics An Augmented Three-Pass System Combination Framework: DCU Combination System for WMT 2010 Jinhua Du, Pavel Pecina, Andy Way CNGL, School of Computing Dublin City University Dublin 9, Ireland {jdu, ppecina, away}@computing.dcu.ie Abstract This paper describes the augmented three- pass system combination framework of the Dublin City University (DCU) MT group for the WMT 2010 system combi- nation task. The basic three-pass frame- work includes building individual confu- sion networks (CNs), a super network, and a modified Minimum Bayes-risk (mCon- MBR) decoder. The augmented parts for WMT2010 tasks include 1) a rescoring component which is used to re-rank the N-best lists generated from the individual CNs and the super network, 2) a new hy- pothesis alignment metric – TERp – that is used to carry out English-targeted hy- pothesis alignment, and 3) more differ- ent backbone-based CNs which are em- ployed to increase the diversity of the mConMBR decoding phase. We took part in the combination tasks of English- to-Czech and French-to-English. Exper- imental results show that our proposed combination framework achieved 2.17 ab- solute points (13.36 relative points) and 1.52 absolute points (5.37 relative points) in terms of BLEU score on English-to- Czech and French-to-English tasks re- spectively than the best single system. We also achieved better performance on hu- man evaluation. 1 Introduction In several recent years, system combination has become not only a research focus, but also a pop- ular evaluation task due to its help in improving machine translation quality. Generally, most com- bination approaches are based on a confusion net- work (CN) which can effectively re-shuffle the translation hypotheses and generate a new target sentence. A CN is essentially a directed acyclic graph built from a set of translation hypotheses against a reference or “backbone”. Each arc be- tween two nodes in the CN denotes a word or to- ken, possibly a null item, with an associated pos- terior probability. Typically, the dominant CN is constructed at the word level by a state-of-the-art framework: firstly, a minimum Bayes-risk (MBR) decoder (Kumar and Byrne, 2004) is utilised to choose the back- bone from a merged set of hypotheses, and then the remaining hypotheses are aligned against the backbone by a specific alignment approach. Cur- rently, most research in system combination has focused on hypothesis alignment due to its signif- icant influence on combination quality. A multiple CN or “super-network” framework was firstly proposed in Rosti et al. (2007) who used each of all individual system results as the backbone to build CNs based on the same align- ment metric, TER (Snover et al., 2006). A consen- sus network MBR (ConMBR) approach was pre- sented in (Sim et al., 2007), where MBR decod- ing is employed to select the best hypothesis with the minimum cost from the original single system outputs compared to the consensus output. Du and Way (2009) proposed a combination strategy …

Content truncated to 500 words