LIMSI‘s Statistical Translation Systems for WMT‘09

Proceedings of the Fourth Workshop on Statistical Machine Translation , pages 100–104, Athens, Greece, 30 March – 31 March 2009. c⃝2009 Association for Computational Linguistics LIMSI’s statistical translation systems for WMT’09 Alexandre Allauzen, Josep Crego, Aurélien Max and François Yvon LIMSI/CNRS and Université Paris-Sud 11, France BP 133, 91403 Orsay Cédex firstname.lastname@limsi.fr Abstract This paper describes our Statistical Ma- chine Translation systems for the WMT09 (en:fr) shared task. For this evaluation, we have developed four systems, using two different MT Toolkits: our primary sub- mission, in both directions, is based on Moses, boosted with contextual informa- tion on phrases, and is contrasted with a conventional Moses-based system. Addi- tional contrasts are based on the Ncode toolkit, one of which uses (part of) the En- glish/French GigaWord parallel corpus. 1 Introduction This paper describes our Statistical Machine Translation systems for the WMT09 (en:fr) shared task. For this evaluation, we have developed four systems, using two different MT toolkits: our primary submission, in both direction, is based on Moses, boosted with contextual information on phrases; we also provided a contrast with a vanilla Moses-based system. Additional contrasts are based on the N-code decoder, one of which takes advantage of (part of) the English/French Gi- gaWord parallel corpus. 2 System architecture and resources In this section, we describe the main characteris- tics of the baseline phrase-based systems used in this evaluation and the resources that were used to train our models. 2.1 Pre- and post-processing tools All the available textual corpora were processed and normalized using in-house text processing tools. Our last year experiments (Déchelotte et al., 2008) revealed that using better normalization tools provides a significant reward in BLEU, a fact that we could observe again this year. The down- side is the need to post-process our outputs so as to “detokenize” them for scoring purposes, which is unfortunately an error-prone process. Based again on last year’s experiments, our sys- tems are built in “true case”: the first letter of each sentence is lowercased when it should be, and the remaining tokens are left as is. Finally, the N-code (see 2.5) and the context- aware (see 3) systems require the source to be morpho-syntactically analysed. This was per- formed using the TreeTagger1 for both languages. 2.2 Alignment and translation models Our baseline translation models (see 2.4 and 2.5) use all the parallel corpora distributed for this eval- uation: Europarl V4, news commentary (2006- 2009) and the additional news data, totalling 1.5M sentences. Our preliminary attempts with larger translation models using the GigaWord corpus are reported in section 3.2. All these corpora were aligned with GIZA++2 using default settings. 2.3 Language Models To train our language models (LMs), we took ad- vantage of the a priori information that the test set would be of newspaper/newswire genre. We 1http://www.ims.uni-stuttgart.de/ projekte/corplex/TreeTagger. 2http://www.fjoch.com/GIZA++.html. 100 Source Period M. words News texts 1994-06 3 317 En BN transcripts 2000-07 341 WMT 86 Newswires 1994-07 723 Newspapers 1987-06 486 Fr WEB 2008 23 WMT 46 News-train08 167 Table 1: …

Content truncated to 500 words