Lessons from NRC’s Portage System at WMT 2010

Proceedings of the Joint 5th Workshop on Statistical Machine Translation and MetricsMATR, pages 127–132, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics Lessons from NRC’s Portage System at WMT 2010 Samuel Larkin, Boxing Chen, George Foster, Ulrich Germann, Eric Joanis, Howard Johnson, and Roland Kuhn National Research Council of Canada (NRC) Gatineau, Québec, Canada. Firstname.Lastname@cnrc-nrc.gc.ca Abstract NRC’s Portage system participated in the Eng- lish-French (E-F) and French-English (F-E) translation tasks of the ACL WMT 2010 eval- uation. The most notable improvement over earlier versions of Portage is an efficient im- plementation of lattice MERT. While Portage has typically performed well in Chinese to English MT evaluations, most recently in the NIST09 evaluation, our participation in WMT 2010 revealed some interesting differences be- tween Chinese-English and E-F/F-E transla- tion, and alerted us to certain weak spots in our system. Most of this paper discusses the problems we found in our system and ways of fixing them. We learned several lessons that we think will be of general interest. 1 Introduction Portage, the statistical machine translation sys- tem of the National Research Council of Canada (NRC), is a two-pass phrase-based system. The translation tasks to which it is most often applied are Chinese to English, English to French (hen- ceforth “E-F”), and French to English (hence- forth “F-E”): in recent years we worked on Chi- nese-English translation for the GALE project and for NIST evaluations, and English and French are Canada’s two official languages. In WMT 2010, Portage scored 28.5 BLEU (un- cased) for F-E, but only 27.0 BLEU (uncased) for E-F. For both language pairs, Portage tru- ecasing caused a loss of 1.4 BLEU; other WMT systems typically lost around 1.0 BLEU after truecasing. In Canada, about 80% of translations between English and French are from English to French, so we would have preferred better results for that direction. This paper first describes the version of Portage that participated in WMT 2010. It then analyzes problems with the system and describes the solutions we found for some of them. 2 Portage system description 2.1 Core engine and training data The NRC system uses a standard two-pass phrase-based approach. Major features in the first-pass loglinear model include phrase tables derived from symmetrized IBM2 alignments and symmetrized HMM alignments, a distance-based distortion model, a lexicalized distortion model, and language models (LMs) that can be either static or else dynamic mixtures. Each phrase ta- ble used was a merged one, created by separately training an IBM2-based and an HMM-based joint count table on the same data and then add- ing the counts. Each includes relative frequency estimates and lexical estimates (based on Zens and Ney, 2004) of forward and backward condi- tional probabilities. The lexicalized distortion probabilities are also obtained by adding IBM2 and HMM counts. They involve 6 features (mo- notone, swap and discontinuous features for fol- lowing and preceding phrase) and are condi- tioned on phrase pairs in a model similar to that of Moses (Koehn et al., 2005); a …

Content truncated to 500 words