JHU System Combination Scheme for WMT 2010

Proceedings of the Joint 5th Workshop on Statistical Machine Translation and MetricsMATR, pages 311–314, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics JHU System Combination Scheme for WMT 2010 Sushant Narsale Johns Hopkins University Baltimore, USA. sushant@jhu.edu Abstract This paper describes the JHU system combination scheme that was used in the WMT 2010 submission. The in- cremental alignment scheme of (Karakos et.al, 2008) was used for confusion net- work generation. The system order in the alignment of each sentence was learned using SVMs, following the work of (Karakos et.al, 2010). Additionally, web-scale n-grams from the Google cor- pus were used to build language models that improved the quality of the combi- nation output. Experiments in Spanish- English, French-English, German-English and Czech-English language pairs were conducted, and the results show approxi- mately 1 BLEU point and 2 TER points improvement over the best individual sys- tem. 1 Introduction System Combination refers to the method of com- bining output of multiple MT systems, to pro- duce a output better than each individual system. Currently, there are several approaches to ma- chine translation which can be classified as phrase- based, hierarchical, syntax-based (Hildebrand and Vogel, 2008) which are equally good in their trans- lation quality even though the underlying frame- works are completely different. The motivation behind System Combination arises from this di- versity in the state-of-art MT systems, which sug- gests that systems with different paradigms make different errors, and can be made better by com- bining their strengths. One approach of combining translations is based on representing translations by confusion network and then aligning these confusion net- works using string alignment algorithms (Rosti et.al, 2009), (Karakos and Khudanpur, 2008). Another approach generates features for every translation to train algorithms for ranking systems based on their quality and the top ranking output is considered to be a candidate translation, (Hilde- brand and Vogel, 2008) is an example of ranking based combination. We use ideas from ranking based approaches to learn order in which systems should be aligned in a confusion network based approach. Our approach is based on incremental align- ment of confusion networks (Karakos et.al, 2008), wherein each system output is represented by a confusion network. The confusion networks are then aligned in a pre-defined order to generate a combination output. This paper contributes two enhancements to (Karakos et.al, 2008). First, use of Support Vector Machines to learn order in which the system outputs should be aligned. Sec- ond, we explore use of Google n-grams for build- ing dynamic language model and interpolate the resulting language model with a large static lan- guage model for rescoring of system combination outputs. The rest of the paper is organized as follows: Section 2 illustrates the idea and pipeline of the baseline combination system; Section 3 gives de- tails of SVM ranking for learning system order for combination; Section 4 explains use of Google n-gram based language models; Results are dis- cussed in Section 5; Concluding remarks are given in Section 6; …

Content truncated to 500 words