The CMU-ARK German-English Translation System

Proceedings of the 6th Workshop on Statistical Machine Translation, pages 337–343, Edinburgh, Scotland, UK, July 30–31, 2011. c⃝2011 Association for Computational Linguistics The CMU-ARK German-English Translation System Chris Dyer Kevin Gimpel Jonathan H. Clark Noah A. Smith Language Technologies Institute Carnegie Mellon University Pittsburgh, PA, 15213, USA {cdyer,kgimpel,jhclark,nasmith}@cs.cmu.edu Abstract This paper describes the German-English translation system developed by the ARK re- search group at Carnegie Mellon University for the Sixth Workshop on Machine Trans- lation (WMT11). We present the results of several modeling and training improvements to our core hierarchical phrase-based trans- lation system, including: feature engineering to improve modeling of the derivation struc- ture of translations; better handing of OOVs; and using development set translations into other languages to create additional pseudo- references for training. 1 Introduction We describe the German-English translation system submitted to the shared translation task in the Sixth Workshop on Machine Translation (WMT11) by the ARK research group at Carnegie Mellon Univer- sity.1 The core translation system is a hierarchical phrase-based machine translation system (Chiang, 2007) that has been extended in several ways de- scribed in this paper. Some of our innovations focus on modeling. Since German and English word orders can diverge considerably, particularly in non-matrix clauses, we focused on feature engineering to improve the modeling of long-distance relationships, which are poorly captured in standard hierarchical phrase- based translation models. To do so, we devel- oped features that assess the goodness of the source 1http://www.ark.cs.cmu.edu language parse tree under the translation grammar (rather than of a “linguistic” grammar). To train the feature weights, we made use of a novel two-phase training algorithm that incorporates a probabilistic training objective and standard minimum error train- ing (Och, 2003). These segmentation features were supplemented with a 7-gram class-based language model, which more directly models long-distance relationships. Together, these features provide a modest improvement over the baseline and suggest interesting directions for future work. While our work on parse modeling was involved and required substantial changes to the training pipeline, some other modeling enhancements were quite simple: for example, improving how out-of-vocabulary words are handled. We propose a very simple change, and show that it provides a small, consistent gain. On the training side, we had two improvements over our baseline system. First, we were inspired by the work of Madnani (2010), who showed that when training to optimize BLEU (Papineni et al., 2002), overfitting is reduced by supplementing a sin- gle human-generated reference translation with ad- ditional computer-generated references. We gener- ated supplementary pseudo-references for our de- velopment set (which is translated into many lan- guages, but once) by using MT output from a sec- ondary Spanish-English translation system. Second, following Foster and Kuhn (2009), we used a sec- ondary development set to select from among many optimization runs, which further improved general- ization. We largely sought techniques that did not require language-specific resources (e.g., treebanks, POS 337 annotations, morphological analyzers). An excep- tion is a compound segmentation model used for preprocessing that was trained on a …

Content truncated to 500 words