An Iteratively-Trained Segmentation-Free Phrase Translation Model for Statistical Machine Translation

Proceedings of the Second Workshop on Statistical Machine Translation, pages 112–119, Prague, June 2007. c⃝2007 Association for Computational Linguistics An Iteratively-Trained Segmentation-Free Phrase Translation Model for Statistical Machine Translation Robert C. Moore Chris Quirk Microsoft Research Redmond, WA 98052, USA {bobmoore,chrisq}@microsoft.com Abstract Attempts to estimate phrase translation probablities for statistical machine transla- tion using iteratively-trained models have repeatedly failed to produce translations as good as those obtained by estimating phrase translation probablities from surface statis- tics of bilingual word alignments as de- scribed by Koehn, et al. (2003). We pro- pose a new iteratively-trained phrase trans- lation model that produces translations of quality equal to or better than those pro- duced by Koehn, et al.’s model. Moreover, with the new model, translation quality de- grades much more slowly as pruning is tigh- tend to reduce translation time. 1 Introduction Estimates of conditional phrase translation probabil- ities provide a major source of translation knowl- edge in phrase-based statistical machine translation (SMT) systems. The most widely used method for estimating these probabilities is that of Koehn, et al. (2003), in which phrase pairs are extracted from word-aligned bilingual sentence pairs, and their translation probabilities estimated heuristically from surface statistics of the extracted phrase pairs. We will refer to this approach as “the standard model”. There have been several attempts to estimate phrase translation probabilities directly, using gen- erative models trained iteratively on a parallel cor- pus using the Expectation Maximization (EM) algo- rithm. The first of these models, that of Marcu and Wong (2002), was found by Koehn, et al. (2003), to produce translations not quite as good as their method. Recently, Birch et al. (2006) tried the Marcu and Wong model constrained by a word alignment and also found that Koehn, et al.’s model worked better, with the advantage of the standard model increasing as more features were added to the overall translation model. DeNero et al. (2006) tried a different generative phrase translation model anal- ogous to IBM word-translation Model 3 (Brown et al., 1993), and again found that the standard model outperformed their generative model. DeNero et al. (2006) attribute the inferiority of their model and the Marcu and Wong model to a hid- den segmentation variable, which enables the EM algorithm to maximize the probability of the train- ing data without really improving the quality of the model. We propose an iteratively-trained phrase translation model that does not require different seg- mentations to compete against one another, and we show that this produces translations of quality equal to or better than those produced by the standard model. We find, moreover, that with the new model, translation quality degrades much more slowly as pruning is tightend to reduce translation time. Decoding efficiency is usually considered only in the design and implementation of decoding algo- rithms, or the choice of model structures to support faster decoding algorithms. We are not aware of any attention previously having been paid to the effect of different methods of parameter estimation on trans- lation …

Content truncated to 500 words