Training Phrase Translation Models with Leaving-One-Out

Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 475–484, Uppsala, Sweden, 11-16 July 2010. c⃝2010 Association for Computational Linguistics Training Phrase Translation Models with Leaving-One-Out Joern Wuebker and Arne Mauser and Hermann Ney Human Language Technology and Pattern Recognition Group RWTH Aachen University, Germany <surname>@cs.rwth-aachen.de Abstract Several attempts have been made to learn phrase translation probabilities for phrase- based statistical machine translation that go beyond pure counting of phrases in word-aligned training data. Most approaches report problems with over- fitting. We describe a novel leaving- one-out approach to prevent over-fitting that allows us to train phrase models that show improved translation performance on the WMT08 Europarl German-English task. In contrast to most previous work where phrase models were trained sepa- rately from other models used in transla- tion, we include all components such as single word lexica and reordering mod- els in training. Using this consistent training of phrase models we are able to achieve improvements of up to 1.4 points in BLEU. As a side effect, the phrase table size is reduced by more than 80%. 1 Introduction A phrase-based SMT system takes a source sen- tence and produces a translation by segmenting the sentence into phrases and translating those phrases separately (Koehn et al., 2003). The phrase trans- lation table, which contains the bilingual phrase pairs and the corresponding translation probabil- ities, is one of the main components of an SMT system. The most common method for obtain- ing the phrase table is heuristic extraction from automatically word-aligned bilingual training data (Och et al., 1999). In this method, all phrases of the sentence pair that match constraints given by the alignment are extracted. This includes over- lapping phrases. At extraction time it does not matter, whether the phrases are extracted from a highly probable phrase alignment or from an un- likely one. Phrase model probabilities are typically defined as relative frequencies of phrases extracted from word-aligned parallel training data. The joint counts C( ˜f, ˜e) of the source phrase ˜f and the tar- get phrase ˜e in the entire training data are normal- ized by the marginal counts of source and target phrase to obtain a conditional probability pH( ˜f|˜e) = C( ˜f, ˜e) C(˜e) . (1) The translation process is implemented as a weighted log-linear combination of several mod- els hm(eI 1, sK 1 , fJ 1 ) including the logarithm of the phrase probability in source-to-target as well as in target-to-source direction. The phrase model is combined with a language model, word lexicon models, word and phrase penalty, and many oth- ers. (Och and Ney, 2004) The best translation ˆeˆI 1 as defined by the models then can be written as ˆe ˆI 1 = argmax I,eI 1 � M � m=1 λmhm(eI 1, sK 1 , fJ 1 ) � (2) In this work, we propose to directly train our phrase models by applying a forced alignment pro- cedure where we use the decoder to find a phrase alignment between …

Content truncated to 500 words