Building a Statistical Machine Translation System for French Using the Europarl Corpus

Proceedings of the Second Workshop on Statistical Machine Translation, pages 189–192, Prague, June 2007. c⃝2007 Association for Computational Linguistics Building a Statistical Machine Translation System for French using the Europarl Corpus Holger Schwenk LIMSI-CNRS, bat 508, BP 133 91403 Orsay cedex, FRANCE schwenk@limsi.fr Abstract This paper describes the development of a statistical machine translation system based on the Moses decoder for the 2007 WMT shared tasks. Several different translation strategies were explored. We also use a sta- tistical language model that is based on a continuous representation of the words in the vocabulary. By these means we expect to take better advantage of the limited amount of training data. Finally, we have investi- gated the usefulness of a second reference translation of the development data. 1 Introduction This paper describes the development of a statistical machine translation system based on the Moses de- coder (Koehn et al., 2007) for the 2007 WMT shared tasks. Due to time constraints, we only considered the translation between French and English. A sys- tem with a similar architecture was successfully ap- plied to the translation between Spanish and En- glish in the framework of the 2007 TC-STAR eval- uation.1 For the 2007 WMT shared task a recipe is provided to build a baseline translation system using the Moses decoder. Our system differs in several as- pects from this base-line: 1) the training data is not lower-cased; 2) Giza alignments are calculated on sentences of up to 90 words; 3) a two pass-decoding was used; and 4) a so called continuous space lan- guage model is used in order to take better advantage of the limited amount of training data. 1A paper on this work is submitted to MT Sumit 2007. This architecture is motivated and detailed in the following sections. 2 Architecture of the system The goal of statistical machine translation (SMT) is to produce a target sentence e from a source sen- tence f. It is today common practice to use phrases as translation units (Koehn et al., 2003; Och and Ney, 2003) and a log linear framework in order to introduce several models explaining the translation process: e∗ = arg max p(e|f) = arg max e {exp( � i λihi(e, f))} (1) The feature functions hi are the system models and the λi weights are typically optimized to maximize a scoring function on a development set (Och and Ney, 2002). In our system fourteen features func- tions were used, namely phrase and lexical transla- tion probabilities in both directions, seven features for the lexicalized distortion model, a word and a phrase penalty and a target language model (LM). The system is constructed as follows. First, Giza++ is used to perform word alignments in both directions. Second, phrases and lexical reorderings are extracted using the default settings of the Moses SMT toolkit. A target LM is then constructed as detailed in section 2.1. The translation itself is per- formed in two passes: first, Moses in run and a 1000- best list is generated …

Content truncated to 500 words