Toward Using Morphology in French-English Phrase-Based SMT

Proceedings of the Fourth Workshop on Statistical Machine Translation , pages 150–154, Athens, Greece, 30 March – 31 March 2009. c⃝2009 Association for Computational Linguistics Toward Using Morphology in French-English Phrase-based SMT Marine Carpuat Center for Computational Learning Systems Columbia University 475 Riverside Drive, New York, NY 10115 marine@ccls.columbia.edu Abstract We describe the system used in our sub- mission to the WMT-2009 French-English translation task. We use the Moses phrase- based Statistical Machine Translation sys- tem with two simple modications of the decoding input and word-alignment strat- egy based on morphology, and analyze their impact on translation quality. 1 Introduction In this first participation to the French-English translation task at WMT, our goal was to build a standard phrase-based statistical machine transla- tion system and study the impact of French mor- phological variations at different stages of training and decoding. Many strategies have been proposed to inte- grate morphology information in SMT, including factored translation models (Koehn and Hoang, 2007), adding a translation dictionary containing inflected forms to the training data (Schwenk et al., 2008), entirely replacing surface forms by representations built on lemmas and POS tags (Popovi´c and Ney, 2004), morphemes learned in an unsupervised manner (Virpojia et al., 2007), and using Porter stems and even 4-letter prefixes for word alignment (Watanabe et al., 2006). In non-European languages, such as Arabic, heavy effort has been put in identifying appropriate in- put representations to improve SMT quality (e.g., Sadat and Habash (2006)) As a first step toward using morphology infor- mation in our French-English SMT system, this submission focused on studying the impact of ∗The author was partially funded by GALE DARPA Con- tract No. HR0011-06-C-0023. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the Defense Advanced Research Projects Agency. different input representations for French based on the POS and lemmatization provided by the Treetagger tool (Schmid, 1994). In the WMT09 French-English data sets, we observe that more than half of the words that are unknown in the translation lexicon actually occur in the training data under different inflected forms. We show that combining a lemma backoff strategy at decoding time and improving alignments by generalizing across verb surface forms improves OOV rates and translation quality. 2 Translation system 2.1 Data sets We use a subset of the data made available for the official French to English translation task. The evaluation test set consists of French news data from September to October 2008, however the bulk of the training data is not from the same do- main. The translation model was trained on the Europarl corpus (europarl-v4) and the small news commentary corpus (news-commentary09). Fol- lowing D´echelotte et al. (2008), we learn a sin- gle phrase table and reordering model rather than one for each domain, as it was found to yield bet- ter performance in a very similar setting. The language model was trained on the English side of these parallel corpora augmented …

Content truncated to 500 words