BBN System Description for WMT10 System Combination Task

Proceedings of the Joint 5th Workshop on Statistical Machine Translation and MetricsMATR, pages 321–326, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics BBN System Description for WMT10 System Combination Task Antti-Veikko I. Rosti and Bing Zhang and Spyros Matsoukas and Richard Schwartz Raytheon BBN Technologies, 10 Moulton Street, Cambridge, MA 02138, USA {arosti,bzhang,smatsouk,schwartz}@bbn.com Abstract BBN submitted system combination out- puts for Czech-English, German-English, Spanish-English, French-English, and All- English language pairs. All combinations were based on confusion network decod- ing. An incremental hypothesis alignment algorithm with flexible matching was used to build the networks. The bi-gram de- coding weights for the single source lan- guage translations were tuned directly to maximize the BLEU score of the decod- ing output. Approximate expected BLEU was used as the objective function in gra- dient based optimization of the combina- tion weights for a 44 system multi-source language combination (All-English). The system combination gained around 0.4- 2.0 BLEU points over the best individual systems on the single source conditions. On the multi-source condition, the system combination gained 6.6 BLEU points. 1 Introduction The BBN submissions to the WMT10 system combination task were based on confusion net- work decoding. The confusion networks were built using the incremental hypothesis alignment algorithm with flexible matching introduced in the BBN submission for the WMT09 system combi- nation task (Rosti et al., 2009). This year, the system combination weights were tuned to max- imize the BLEU score (Papineni et al., 2002) of the 1-best decoding output (lattice based BLEU tuning) using downhill simplex method (Press et al., 2007). A 44 system multi-source combina- tion was also submitted. Since the gradient-free optimization algorithms do not seem to be able to handle more than 20-30 weights, a gradient ascent to maximize an approximate expected BLEU ob- jective was used to optimize the larger number of weights. The lattice based BLEU tuning may be imple- mented using any optimization algorithm that does not require the gradient of the objective function. Due to the size of the lattices, the objective func- tion evaluation may have to be distributed to mul- tiple servers. The optimizer client accumulates the BLEU statistics of the 1-best hypotheses from the servers for given search weights, computes the fi- nal BLEU score, and passes it to the optimiza- tion algorithm which returns a new set of search weights. The lattice based tuning explores the en- tire search space and does not require multiple de- coding iterations with N-best list merging to ap- proximate the search space as in the standard min- imum error rate training (Och, 2003). This allows much faster turnaround in weight tuning. Differentiable approximations of BLEU have been proposed for consensus decoding. Tromble et al. (2008) used a linear approximation and Pauls et al. (2009) used a closer approximation called CoBLEU. CoBLEU is based on the BLEU for- mula but the n-gram counts are replaced by ex- pected counts over a translation forest. Due to the min-functions required in converting the n-gram counts to matches …

Content truncated to 500 words