A Discriminative Latent Variable Model for Statistical Machine Translation

Proceedings of ACL-08: HLT, pages 200–208, Columbus, Ohio, USA, June 2008. c⃝2008 Association for Computational Linguistics A Discriminative Latent Variable Model for Statistical Machine Translation Phil Blunsom, Trevor Cohn and Miles Osborne School of Informatics, University of Edinburgh 2 Buccleuch Place, Edinburgh, EH8 9LW, UK {pblunsom,tcohn,miles}@inf.ed.ac.uk Abstract Large-scale discriminative machine transla- tion promises to further the state-of-the-art, but has failed to deliver convincing gains over current heuristic frequency count systems. We argue that a principle reason for this failure is not dealing with multiple, equivalent transla- tions. We present a translation model which models derivations as a latent variable, in both training and decoding, and is fully discrimina- tive and globally optimised. Results show that accounting for multiple derivations does in- deed improve performance. Additionally, we show that regularisation is essential for max- imum conditional likelihood models in order to avoid degenerate solutions. 1 Introduction Statistical machine translation (SMT) has seen a resurgence in popularity in recent years, with progress being driven by a move to phrase-based and syntax-inspired approaches. Progress within these approaches however has been less dramatic. We be- lieve this is because these frequency count based1 models cannot easily incorporate non-independent and overlapping features, which are extremely use- ful in describing the translation process. Discrimi- native models of translation can include such fea- tures without making assumptions of independence or explicitly modelling their interdependence. How- ever, while discriminative models promise much, they have not been shown to deliver significant gains 1We class approaches using minimum error rate training (Och, 2003) frequency count based as these systems re-scale a handful of generative features estimated from frequency counts and do not support large sets of non-independent features. over their simpler cousins. We argue that this is due to a number of inherent problems that discrimina- tive models for SMT must address, in particular the problems of spurious ambiguity and degenerate so- lutions. These occur when there are many ways to translate a source sentence to the same target sen- tence by applying a sequence of steps (a derivation) of either phrase translations or synchronous gram- mar rules, depending on the type of system. Exist- ing discriminative models require a reference deriva- tion to optimise against, however no parallel cor- pora annotated for derivations exist. Ideally, a model would account for this ambiguity by marginalising out the derivations, thus predicting the best transla- tion rather than the best derivation. However, doing so exactly is NP-complete. For this reason, to our knowledge, all discriminative models proposed to date either side-step the problem by choosing simple model and feature structures, such that spurious am- biguity is lessened or removed entirely (Ittycheriah and Roukos, 2007; Watanabe et al., 2007), or else ig- nore the problem and treat derivations as translations (Liang et al., 2006; Tillmann and Zhang, 2007). In this paper we directly address the problem of spurious ambiguity in discriminative models. We use a synchronous context free grammar (SCFG) translation system (Chiang, 2007), a model which has yielded state-of-the-art results on many …

Content truncated to 500 words