MANY: Open Source MT System Combination at WMT’10

Proceedings of the Joint 5th Workshop on Statistical Machine Translation and MetricsMATR, pages 271–275, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics MANY : Open Source MT System Combination at WMT’10 Lo¨ıc Barrault LIUM, University of Le Mans Le Mans, France. FirstName.LastName@lium.univ-lemans.fr Abstract LIUM participated in the System Combi- nation task of the Fifth Workshop on Sta- tistical Machine Translation (WMT 2010). Hypotheses from 5 French/English MT systems were combined with MANY, an open source system combination software based on confusion networks currently de- veloped at LIUM. The system combination yielded signifi- cant improvements in BLEU score when applied on WMT’09 data. The same be- havior has been observed when tuning is performed on development data of this year evaluation. 1 Introduction This year, the LIUM computer science labora- tory has participated in the French-English sys- tem combination task at WMT’10 evaluation cam- paign. The system used for this task is MANY1 (Barrault, 2010), an open source system combina- tion software based on Confusion Networks (CN). Several improvements have been made in order to being able to combine many systems outputs in a decent time. The focus has been put on the tuning step, and more precisely how to perform system parameter tuning. Two methods have been experimented cor- responding to two different representations of sys- tem combination. In the first one, system combi- nation is considered as a whole : fed by system hypotheses as input and generating a new hypoth- esis as output. The second method considers that the alignment module is independent from the de- coder, so that the parameters from each module can be tuned separately. 1MANY is available at the following address http:// www-lium.univ-lemans.fr/˜barrault/MANY Those tuning approaches are described in sec- tion 3. Before that, a quick description of MANY, including recent developments, can be found in section 2. Results on WMT’09 data are pre- sented in section 4 along results of tuning on newssyscombtune2010. 2 System description MANY is a system combination software (Bar- rault, 2010) based on the decoding of a lattice made of several Confusion Networks (CN). This is a widespread approach in MT system combination (Rosti et al., 2007); (Shen et al., 2008); (Karakos et al., 2008). MANY can be decomposed in two main modules. The first one is the alignment mod- ule which actually is a modified version of TERp (Snover et al., 2009). Its role is to incrementally align the hypotheses against a backbone in order to create a confusion network. Those confusion net- works are then connected together to create a lat- tice. This module uses different costs (which cor- responds to a match, an insertion, a deletion, a sub- stitution, a shift, a synonym and a stem) to com- pute the best alignment and incrementally build a confusion network. In the case of confusion network, the match (substitution, synonyms, and stems) costs are considered when the word in the hypothesis matches (is a substitution, a synonyms or a stems of) at least one word of …

Content truncated to 500 words