Accuracy-Based Scoring for DOT: Towards Direct Error Minimization for Data-Oriented Translation

Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 371–380, Singapore, 6-7 August 2009. c⃝2009 ACL and AFNLP Accuracy-Based Scoring for DOT: Towards Direct Error Minimization for Data-Oriented Translation Daniel Galron CIMS New York University galron@cs.nyu.edu Sergio Penkale, Andy Way CNGL Dublin City University {spenkale,away} @computing.dcu.ie I. Dan Melamed AT&T Shannon Laboratory {lastname} @research.att.com Abstract In this work we present a novel technique to rescore fragments in the Data-Oriented Translation model based on their contri- bution to translation accuracy. We de- scribe three new rescoring methods, and present the initial results of a pilot experi- ment on a small subset of the Europarl cor- pus. This work is a proof-of-concept, and is the first step in directly optimizing trans- lation decisions solely on the hypothesized accuracy of potential translations resulting from those decisions. 1 Introduction The Data-Oriented Translation (DOT) (Poutsma, 2000) model is a tree-structured translation model, in which linked subtree fragments extracted from a parsed bitext are composed to cover a source- language sentence to be translated. Each linked fragment pair consists of a source-language side and a target-language side, similar to (Wu, 1997). Translating a new sentence involves composing the linked fragments into derivations so that a new source-language sentence is covered by the source tree fragments of the linked pairs, where the yields of the target-side derivations are the can- didate translations. Derivations are scored accord- ing to their likelihood, and the translation is se- lected from the derivation pair with the highest score. However, we have no reason to believe that maximizing likelihood is the best way to maxi- mize translation accuracy – likelihood and accu- racy do not necessarily correlate well. We can frame the problem as a search problem, where we are searching a space of derivations for the one that yields the highest scoring translation. By putting weights on the derivations in the search space, we wish to point the decoder in the direc- tion of the optimal translation. Since we want the decoder to find the translation with the high- est evaluation score, we would want to score the derivations with weights that correlate well with the particular evaluation measure in mind. Much of the work in the MT literature has focused on the scoring of translation decisions made. (Yamada and Knight, 2001) follow (Brown et al., 1993) in using the noisy channel model, by decomposing the translation decisions mod- eled by the translation model into different types, and inducing probability distributions via max- imum likelihood estimation over each decision type. This model is then decoded as described in (Yamada and Knight, 2002). This type of ap- proach is also followed in (Galley et al., 2006). There has been some previous work on accuracy-driven training techniques for SMT, such as MERT (Och, 2003) and the Simplex Armijo Downhill method (Zhao and Chen, 2009), which tune the parameters in a linear combination of var- ious phrase scores according to a held-out tun- ing set. While this does tune the …

Content truncated to 500 words