EMDC: A Semi-supervised Approach for Word Alignment

Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 349–357, Beijing, August 2010 EMDC: A Semi-supervised Approach for Word Alignment Qin Gao Language Technologies Institute Carnegie Mellon University qing@cs.cmu.edu Francisco Guzman Centro de Sistemas Inteligentes Tecnol´ogico de Monterrey guzmanhe@gmail.com Stephan Vogel Language Technologies Institute Carnegie Mellon University stephan.vogel@cs.cmu.edu Abstract This paper proposes a novel semi- supervised word alignment technique called EMDC that integrates discrimina- tive and generative methods. A discrim- inative aligner is used to find high preci- sion partial alignments that serve as con- straints for a generative aligner which implements a constrained version of the EM algorithm. Experiments on small-size Chinese and Arabic tasks show consistent improvements on AER. We also experi- mented with moderate-size Chinese ma- chine translation tasks and got an aver- age of 0.5 point improvement on BLEU scores across five standard NIST test sets and four other test sets. 1 Introduction Word alignment is a crucial component in sta- tistical machine translation (SMT). From a Ma- chine Learning perspective, the models for word alignment can be roughly categorized as gener- ative models and discriminative models. The widely used word alignment tool, i.e. GIZA++ (Och and Ney, 2003), implements the well-known IBM models (Brown et al., 1993) and the HMM model (Vogel et al., 1996), which are genera- tive models. For language pairs such as Chinese- English, the word alignment quality is often un- satisfactory. There has been increasing interest on using manual alignments in word alignment tasks, which has resulted in several discriminative mod- els. Ittycheriah and Roukos (2005) proposed to use only manual alignment links in a maximum entropy model, which is considered supervised. Also, a number of semi-supervised word align- ers have been proposed (Taskar et al., 2005; Liu et al., 2005; Moore, 2005; Blunsom and Cohn, 2006; Niehues and Vogel, 2008). These methods use held-out manual alignments to tune weights for discriminative models, while using the model parameters, model scores or alignment links from unsupervised word aligners as features. Callison- Burch et. al. (2004) proposed a method to interpo- late the parameters estimated by sentence-aligned and word-aligned corpus. Also, there are recent attempts to combine multiple alignment sources using alignment confidence measures so as to im- prove the alignment quality (Huang, 2009). In this paper, the question we address is whether we can jointly improve discriminative models and generative models by feeding the in- formation we get from the discriminative aligner back into the generative aligner. Examples of this line of research include Model 6 (Och and Ney, 2003) and the EMD training approach pro- posed by Fraser and Marcu (2006) and its ex- tension called LEAF aligner (Fraser and Marcu, 2007). These approaches use labeled data to tune additional parameters to weight different compo- nents of the IBM models such as the lexical trans- lation model, the distortion model and the fertility model. These methods are proven to be effective in improving the quality of alignments. However, the discriminative training in these methods is re- stricted in …

Content truncated to 500 words