Discriminative Reranking for Machine Translation

Discriminative Reranking for Machine Translation Libin Shen Dept. of Comp. & Info. Science Univ. of Pennsylvania Philadelphia, PA 19104 libin@seas.upenn.edu Anoop Sarkar School of Comp. Science Simon Fraser Univ. Burnaby, BC V5A 1S6 anoop@cs.sfu.ca Franz Josef Och Info. Science Institute Univ. of Southern California Marina del Rey, CA 90292 och@isi.edu Abstract This paper describes the application of discrim- inative reranking techniques to the problem of machine translation. For each sentence in the source language, we obtain from a baseline sta- tistical machine translation system, a ranked � - best list of candidate translations in the target language. We introduce two novel perceptron- inspired reranking algorithms that improve on the quality of machine translation over the baseline system based on evaluation using the BLEU metric. We provide experimental results on the NIST 2003 Chinese-English large data track evaluation. We also provide theoretical analysis of our algorithms and experiments that verify that our algorithms provide state-of-the- art performance in machine translation. 1 Introduction The noisy-channel model (Brown et al., 1990) has been the foundation for statistical machine translation (SMT) for over ten years. Recently so-called reranking tech- niques, such as maximum entropy models (Och and Ney, 2002) and gradient methods (Och, 2003), have been ap- plied to machine translation (MT), and have provided significant improvements. In this paper, we introduce two novel machine learning algorithms specialized for the MT task. Discriminative reranking algorithms have also con- tributed to improvements in natural language parsing and tagging performance. Discriminative reranking al- gorithms used for these applications include Perceptron, Boosting and Support Vector Machines (SVMs). In the machine learning community, some novel discriminative ranking (also called ordinal regression) algorithms have been proposed in recent years. Based on this work, in this paper, we will present some novel discriminative reranking techniques applied to machine translation. The reranking problem for natural language is neither a clas- sification problem nor a regression problem, and under certain conditions MT reranking turns out to be quite dif- ferent from parse reranking. In this paper, we consider the special issues of apply- ing reranking techniques to the MT task and introduce two perceptron-like reranking algorithms for MT rerank- ing. We provide experimental results that show that the proposed algorithms achieve start-of-the-art results on the NIST 2003 Chinese-English large data track evaluation. 1.1 Generative Models for MT The seminal IBM models (Brown et al., 1990) were the first to introduce generative models to the MT task. The IBM models applied the sequence learning paradigm well-known from Hidden Markov Models in speech recognition to the problem of MT. The source and tar- get sentences were treated as the observations, but the alignments were treated as hidden information learned from parallel texts using the EM algorithm. This source- channel model treated the task of finding the probability ����� , where � is the translation in the target (English) language for a given source (foreign) sentence � , as two generative probability models: the language model ��� which is a generative probability over candidate transla- …

Content truncated to 500 words