An End-to-End Discriminative Approach to Machine Translation

Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 761–768, Sydney, July 2006. c⃝2006 Association for Computational Linguistics An End-to-End Discriminative Approach to Machine Translation Percy Liang Alexandre Bouchard-Cˆot´e Dan Klein Ben Taskar Computer Science Division, EECS Department University of California at Berkeley Berkeley, CA 94720 {pliang, bouchard, klein, taskar}@cs.berkeley.edu Abstract We present a perceptron-style discriminative ap- proach to machine translation in which large feature sets can be exploited. Unlike discriminative rerank- ing approaches, our system can take advantage of learned features in all stages of decoding. We first discuss several challenges to error-driven discrim- inative approaches. In particular, we explore dif- ferent ways of updating parameters given a training example. We find that making frequent but smaller updates is preferable to making fewer but larger up- dates. Then, we discuss an array of features and show both how they quantitatively increase BLEU score and how they qualitatively interact on spe- cific examples. One particular feature we investi- gate is a novel way to introduce learning into the initial phrase extraction process, which has previ- ously been entirely heuristic. 1 Introduction The generative, noisy-channel paradigm has his- torically served as the foundation for most of the work in statistical machine translation (Brown et al., 1994). At the same time, discriminative meth- ods have provided substantial improvements over generative models on a wide range of NLP tasks. They allow one to easily encode domain knowl- edge in the form of features. Moreover, param- eters are tuned to directly minimize error rather than to maximize joint likelihood, which may not correspond well to the task objective. In this paper, we present an end-to-end dis- criminative approach to machine translation. The proposed system is phrase-based, as in Koehn et al. (2003), but uses an online perceptron training scheme to learn model parameters. Unlike mini- mum error rate training (Och, 2003), our system is able to exploit large numbers of specific features in the same manner as static reranking systems (Shen et al., 2004; Och et al., 2004). However, unlike static rerankers, our system does not rely on a baseline translation system. Instead, it up- dates based on its own n-best lists. As parameter estimates improve, the system produces better n- best lists, which can in turn enable better updates in future training iterations. In this paper, we fo- cus on two aspects of the problem of discrimina- tive translation: the inherent difficulty of learning from reference translations, and the challenge of engineering effective features for this task. Discriminative learning from reference transla- tions is inherently problematic because standard discriminative methods need to know which out- puts are correct and which are not. However, a proposed translation that differs from a reference translation need not be incorrect. It may differ in word choice, literalness, or style, yet be fully acceptable. Pushing our system to avoid such al- ternate translations is undesirable. On the other hand, even if a system produces a reference trans- lation, it may …

Content truncated to 500 words