LIMSI’s Statistical Translation Systems for WMT’10

Proceedings of the Joint 5th Workshop on Statistical Machine Translation and MetricsMATR, pages 54–59, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics LIMSI’s statistical translation systems for WMT’10 Alexandre Allauzen, Josep M. Crego, ˙Ilknur Durgar El-Kahlout and Franc¸ois Yvon LIMSI/CNRS and Universit´e Paris-Sud 11, France BP 133, 91403 Orsay Cedex Firstname.Lastname@limsi.fr Abstract This paper describes our Statistical Ma- chine Translation systems for the WMT10 evaluation, where LIMSI participated for two language pairs (French-English and German-English, in both directions). For German-English, we concentrated on nor- malizing the German side through a proper preprocessing, aimed at reducing the lex- ical redundancy and at splitting complex compounds. For French-English, we stud- ied two extensions of our in-house N-code decoder: firstly, the effect of integrating a new bilingual reordering model; second, the use of adaptation techniques for the translation model. For both set of exper- iments, we report the improvements ob- tained on the development and test data. 1 Introduction LIMSI took part in the WMT 2010 evalua- tion campaign and developed systems for two languages pairs: French-English and German- English in both directions. For German-English, we focused on preprocessing issues and performed a series of experiments aimed at normalizing the German side by removing some of the lexical re- dundancy and by splitting compounds. For this pair, all the experiments were performed using the Moses decoder (Koehn et al., 2007). For French- English, we studied two extensions of our n-gram based system: first, the effect of integrating a new bilingual reordering model; second, the use of adaptation techniques for the translation model. Decoding is performed using our in-house N-code (Mari˜no et al., 2006) decoder. 2 System architecture and resources In this section, we describe the main characteris- tics of the phrase-based systems developed for this evaluation and the resources that were used to train our models. As far as resources go, we used all the data supplied by the 2010 evaluation organizers. Based on our previous experiments (D´echelotte et al., 2008) which have demonstrated that better nor- malization tools provide better BLEU scores (Pap- ineni et al., 2002), we took advantage of our in- house text processing tools for the tokenization and detokenization steps. Only for German data did we used the TreeTagger (Schmid, 1994) tok- enizer. Similar to last year’s experiments, all of our systems are built in ”true-case”. 3 German-English systems As German is morphologically more complex than English, the default policy which consists in treat- ing each word form independently from the oth- ers is plagued with data sparsity, which poses a number of difficulties both at training and de- coding time. When aligning parallel texts at the word level, German compound words typi- cally tend to align with more than one English word; this, in turn, tends to increase the number of possible translation counterparts for each En- glish type, and to make the corresponding align- ment scores less reliable. In decoding, new com- pounds or unseen morphological variants of ex- isting words artificially increase the number …

Content truncated to 500 words