Improved Discriminative Bilingual Word Alignment

Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 513–520, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Improved Discriminative Bilingual Word Alignment Robert C. Moore Wen-tau Yih Microsoft ResearchAndreas Bode Redmond, WA 98052, USA {bobmoore,scottyhi,abode}@microsoft.com Abstract For many years, statistical machine trans- lation relied on generative models to pro- vide bilingual word alignments. In 2005, several independent efforts showed that discriminative models could be used to enhance or replace the standard genera- tive approach. Building on this work, we demonstrate substantial improvement in word-alignment accuracy, partly though improved training methods, but predomi- nantly through selection of more and bet- ter features. Our best model produces the lowest alignment error rate yet reported on Canadian Hansards bilingual data. 1 Introduction Until recently, almost all work in statistical ma- chine translation was based on word alignments obtained from combinations of generative prob- abalistic models developed at IBM by Brown et al. (1993), sometimes augmented by an HMM- based model or Och and Ney’s “Model 6” (Och and Ney, 2003). In 2005, however, several in- dependent efforts (Liu et al., 2005; Fraser and Marcu, 2005; Ayan et al., 2005; Taskar et al., 2005; Moore, 2005; Ittycheriah and Roukos, 2005) demonstrated that discriminatively trained models can equal or surpass the alignment accu- racy of the standard models, if the usual unla- beled bilingual training corpus is supplemented with human-annotated word alignments for only a small subset of the training data. The work cited above makes use of various training procedures and a wide variety of features. Indeed, whereas it can be difficult to design a fac- torization of a generative model that incorporates all the desired information, it is relatively easy to add arbitrary features to a discriminative model. We take advantage of this, building on our ex- isting framework (Moore, 2005), to substantially reduce the alignment error rate (AER) we previ- ously reported, given the same training and test data. Through a careful choice of features, and modest improvements in training procedures, we obtain the lowest error rate yet reported for word alignment of Canadian Hansards data. 2 Overall Approach As in our previous work (Moore, 2005), we train two models we call stage 1 and stage 2, both in the form of a weighted linear combination of fea- ture values extracted from a pair of sentences and a proposed word alignment of them. The possible alignment having the highest overall score is se- lected for each sentence pair. Thus, for a sentence pair (e, f) we seek the alignment ˆa such that ˆa = argmaxa n � i=1 λifi(a, e, f) where the fi are features and the λi are weights. The models are trained on a large number of bilin- gual sentence pairs, a small number of which have hand-created word alignments provided to the training procedure. A set of hand alignments of a different subset of the overall training corpus is used to evaluate the models. In the stage 1 model, …

Content truncated to 500 words