LIUM SMT Machine Translation System for WMT 2010

Proceedings of the Joint 5th Workshop on Statistical Machine Translation and MetricsMATR, pages 121–126, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics LIUM SMT Machine Translation System for WMT 2010 Patrik Lambert, Sadaf Abdul-Rauf and Holger Schwenk LIUM, University of Le Mans 72085 Le Mans cedex 9, FRANCE FirstName.LastName@lium.univ-lemans.fr Abstract This paper describes the development of French–English and English–French ma- chine translation systems for the 2010 WMT shared task evaluation. These sys- tems were standard phrase-based statisti- cal systems based on the Moses decoder, trained on the provided data only. Most of our efforts were devoted to the choice and extraction of bilingual data used for training. We filtered out some bilingual corpora and pruned the phrase table. We also investigated the impact of adding two types of additional bilingual texts, ex- tracted automatically from the available monolingual data. We first collected bilin- gual data by performing automatic trans- lations of monolingual texts. The second type of bilingual text was harvested from comparable corpora with Information Re- trieval techniques. 1 Introduction This paper describes the machine translation sys- tems developed by the Computer Science labora- tory at the University of Le Mans (LIUM) for the 2010 WMT shared task evaluation. We only con- sidered the translation between French and En- glish (in both directions). The main differences with respect to previous year’s system (Schwenk et al., 2009) are as follows: restriction to the data recommended for the workshop, usage of the (fil- tered) French–English gigaword bitext, pruning of the phrase table, and usage of automatic trans- lations of the monolingual news corpus to im- prove the translation model. We also used a larger amount of bilingual data extracted from compara- ble corpora than was done in 2009. These different points are described in the rest of the paper, to- gether with a summary of the experimental results showing the impact of each component. 2 Resources Used The following sections describe how the resources provided or allowed in the shared task were used to train the translation and language models of the system. 2.1 Bilingual data Our system was developed in two stages. First, a baseline system was built to generate automatic translations of some of the monolingual data avail- able. These automatic translations may be used directly with the source texts to build additional bitexts, or as queries of an Information Retrieval (IR) system to extract new bitexts from compara- ble corpora. In a second stage, these additional bilingual data were incorporated to the system (see Section 4 and Tables 1 and 2). The latest version of the News-Commentary (NC) corpus, of the Europarl (Eparl) corpus (ver- sion 5), and of the United Nations (UN) corpus were used. We also took as training data a sub- set of the French–English Gigaword (109) cor- pus. Since a significant part of the data was crawled from the web, we thought that many sen- tence pairs may be only approximate translations of each other. We applied a lexical filter to dis- …

Content truncated to 500 words