First Steps towards a General Purpose French/English Statistical Machine Translation System

Proceedings of the Third Workshop on Statistical Machine Translation, pages 119–122, Columbus, Ohio, USA, June 2008. c⃝2008 Association for Computational Linguistics First Steps towards a general purpose French/English Statistical Machine Translation System Holger Schwenk LIUM, University of Le Mans 72085 Le Mans cedex 9 FRANCE schwenk@lium.univ-lemans.fr Jean-Baptiste Fouet Jean Senellart SYSTRAN SA 92044 Paris La D´efense cedex FRANCE fouet,senellart@systran.fr Abstract This paper describes an initial version of a general purpose French/English statistical ma- chine translation system. The main features of this system are the open-source Moses de- coder, the integration of a bilingual dictionary and a continuous space target language model. We analyze the performance of this system on the test data of the WMT’08 evaluation. 1 Introduction Statistical machine translation (SMT) is today con- sidered as a serious alternative to rule-based ma- chine translation (RBMT). While RBMT systems rely on rules and linguistic resources built for that purpose, SMT systems can be developed without the need of any language knowledge and are only based on bilingual sentence-aligned and large mono- lingual data. However, while the monolingual data is usually available in large amounts, bilingual texts are a sparse resource for most of the language pairs. The largest SMT systems are currently build for the translation of Mandarin and Arabic to English, us- ing more than 170M words of bitexts that are eas- ily available from the LDC. Recent human evalua- tions of these systems seem to indicate that they have reached a level of performance allowing a human be- ing to understand the automatic translations and to answer complicated questions on its content (Jones, 2008). In a joint project between the University of Le Mans and the company SYSTRAN, we try to build similar general purpose SMT systems for Euro- pean languages. In the final version, these systems will not only be trained on all available mono- and bilingual data, but also will include additional re- sources from SYSTRAN like high quality dictio- naries, named entity transliteration and rule-based translation of expressions like numbers and dates. Our ultimate goal is to combine the power of data- driven approaches and the concentrated knowledge present in RBMT resources. In this paper, we de- scribe an initial version of an French/English sys- tem and analyze its performance on the test corpora of the WMT’08 workshop. 2 Architecture of the system The goal of statistical machine translation (SMT) is to produce a target sentence e from a source sen- tence f. It is today common practice to use phrases as translation units (Koehn et al., 2003; Och and Ney, 2003) and a log linear framework in order to introduce several models explaining the translation process: e∗ = arg max p(e|f) = arg max e {exp( � i λihi(e, f))} (1) The feature functions hi are the system models and the λi weights are typically optimized to maximize a scoring function on a development set (Och and Ney, 2002). In our system fourteen features func- tions were used, namely phrase and lexical transla- tion …

Content truncated to 500 words