Statistical Phrase-Based Translation

Statistical Phrase-Based Translation Philipp Koehn, Franz Josef Och, Daniel Marcu Information Sciences Institute Department of Computer Science University of Southern California koehn@isi.edu, och@isi.edu, marcu@isi.edu Abstract We propose a new phrase-based translation model and decoding algorithm that enables us to evaluate and compare several, previ- ously proposed phrase-based translation mod- els. Within our framework, we carry out a large number of experiments to understand bet- ter and explain why phrase-based models out- perform word-based models. Our empirical re- sults, which hold for all examined language pairs, suggest that the highest levels of perfor- mance can be obtained through relatively sim- ple means: heuristic learning of phrase trans- lations from word-based alignments and lexi- cal weighting of phrase translations. Surpris- ingly, learning phrases longer than three words and learning phrases from high-accuracy word- level alignment models does not have a strong impact on performance. Learning only syntac- tically motivated phrases degrades the perfor- mance of our systems. 1 Introduction Various researchers have improved the quality of statis- tical machine translation system with the use of phrase translation. Och et al. [1999]’s alignment template model can be reframed as a phrase translation system; Yamada and Knight [2001] use phrase translation in a syntax- based translation system; Marcu and Wong [2002] in- troduced a joint-probability model for phrase translation; and the CMU and IBM word-based statistical machine translation systems1 are augmented with phrase transla- tion capability. Phrase translation clearly helps, as we will also show with the experiments in this paper. But what is the best 1Presentations at DARPA IAO Machine Translation Work- shop, July 22-23, 2002, Santa Monica, CA method to extract phrase translation pairs? In order to investigate this question, we created a uniform evaluation framework that enables the comparison of different ways to build a phrase translation table. Our experiments show that high levels of performance can be achieved with fairly simple means. In fact, for most of the steps necessary to build a phrase-based system, tools and resources are freely available for re- searchers in the field. More sophisticated approaches that make use of syntax do not lead to better performance. In fact, imposing syntactic restrictions on phrases, as used in recently proposed syntax-based translation models [Ya- mada and Knight, 2001], proves to be harmful. Our ex- periments also show, that small phrases of up to three words are sufficient for obtaining high levels of accuracy. Performance differs widely depending on the methods used to build the phrase translation table. We found ex- traction heuristics based on word alignments to be better than a more principled phrase-based alignment method. However, what constitutes the best heuristic differs from language pair to language pair and varies with the size of the training corpus. 2 Evaluation Framework In order to compare different phrase extraction methods, we designed a uniform framework. We present a phrase translation model and decoder that works with any phrase translation table. 2.1 Model The phrase translation model is based on the noisy chan- nel model. We use Bayes rule …

Content truncated to 500 words