Context-Dependent Alignment Models for Statistical Machine Translation

Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the ACL, pages 110–118, Boulder, Colorado, June 2009. c⃝2009 Association for Computational Linguistics Context-Dependent Alignment Models for Statistical Machine Translation Jamie Brunning, Adri`a de Gispert and William Byrne Machine Intelligence Laboratory Department of Engineering, Cambridge University Trumpington Street, Cambridge, CB2 1PZ, U.K. {jjjb2,ad465,wjb31}@eng.cam.ac.uk Abstract We introduce alignment models for Machine Trans- lation that take into account the context of a source word when determining its translation. Since the use of these contexts alone causes data sparsity prob- lems, we develop a decision tree algorithm for clus- tering the contexts based on optimisation of the EM auxiliary function. We show that our context- dependent models lead to an improvement in align- ment quality, and an increase in translation quality when the alignments are used in Arabic-English and Chinese-English translation. 1 Introduction Alignment modelling for Statistical Machine Translation (SMT) is the task of determining translational correspon- dences between the words in pairs of sentences in parallel text. Given a source language word sequence f J 1 and a target language word sequence eI 1, we model the transla- tion probability as P(eI 1|f J 1 ) and introduce a hidden vari- able aI 1 representing a mapping from the target word po- sitions to source word positions such that ei is aligned to fai. Then P(eI 1|f j 1) = � aI 1 P(eI 1, aI 1|f j 1) (Brown et al., 1993). Previous work on statistical alignment modelling has not taken into account the source word context when de- termining translations of that word. It is intuitive that a word in one context, with a particular part-of-speech and particular words surrounding it, may translate differently when in a different context. We aim to take advantage of this information to provide a better estimate of the word’s translation. The challenge of incorporating con- text information is maintaining computational tractability of estimation and alignment, and we develop algorithms to overcome this. The development of efficient estimation procedures for context-dependent acoustic models revolutionised the field of Automatic Speech Recognition (ASR) (Young et al., 1994). Clustering is used extensively for improv- ing parameter estimation of triphone (and higher order) acoustic models, enabling robust estimation of param- eters and reducing the computation required for recog- nition. Kannan et al. (1994) introduce a binary tree- growing procedure for clustering Gaussian models for triphone contexts based on the value of a likelihood ra- tio. We adopt a similar approach to estimate context- dependent translation probabilities. We focus on alignment with IBM Model 1 and HMMs. HMMs are commonly used to generate alignments from which state of the art SMT systems are built. Model 1 is used as an intermediate step in the creation of more pow- erful alignment models, such as HMMs and further IBM models. In addition, it is used in SMT as a feature in Min- imum Error Training (Och et al., 2004) and for rescor- ing lattices of translation hypotheses (Blackwood et …

Content truncated to 500 words