Improving Statistical Word Alignment with Ensemble Methods

R. Dale et al. (Eds.): IJCNLP 2005, LNAI 3651, pp. 462 – 473, 2005. © Springer-Verlag Berlin Heidelberg 2005 Improving Statistical Word Alignment with Ensemble Methods Hua Wu and Haifeng Wang Toshiba (China) Research and Development Center, 5/F., Tower W2, Oriental Plaza, No.1, East Chang An Ave., Dong Cheng District, Beijing, 100738, China {wuhua, wanghaifeng}@rdc.toshiba.com.cn Abstract. This paper proposes an approach to improve statistical word align- ment with ensemble methods. Two ensemble methods are investigated: bagging and cross-validation committees. On these two methods, both weighted voting and unweighted voting are compared under the word alignment task. In addi- tion, we analyze the effect of different sizes of training sets on the bagging method. Experimental results indicate that both bagging and cross-validation committees improve the word alignment results regardless of weighted voting or unweighted voting. Weighted voting performs consistently better than un- weighted voting on different sizes of training sets. 1 Introduction Bilingual word alignment is first introduced as an intermediate result in statistical machine translation (SMT) [3]. Besides being used in SMT, it is also used in transla- tion lexicon building [9], transfer rule learning [10], example-based machine transla- tion [14], etc. In previous alignment methods, some researchers employed statistical word alignment models to build alignment links [3], [4], [8], [11], [16]. Some re- searchers used similarity and association measures to build alignment links [1], [15]. One issue about word alignment is how to improve the performance of a word aligner when the training data are fixed. One possible solution is to use ensemble methods [5], [6]. The ensemble methods were proposed to improve the performance of classifiers. An ensemble of classifiers is a set of classifiers whose individual deci- sions are combined in some way (weighted or unweighted voting) to classify new examples. Many methods for constructing ensembles have been developed [5]. One kind of methods is to resample the training examples. These methods include bagging [2], cross-validation committees [12] and boosting [7]. The two former methods gen- erate the classifiers in parallel while boosting generates the classifiers sequentially. In addition, boosting changes the weights of the training instance that is provided as input to each inducer based on the previously built classifiers. In this paper, we propose an approach to improve word alignment with ensemble methods. Although word alignment is not a classification problem, we can still build different word aligners by resampling the training data. If these aligners perform accurately and diversely on the corpus [6], they can be employed to improve the word alignment results. Here, we investigate two ensemble methods: bagging and Improving Statistical Word Alignment with Ensemble Methods 463 cross-validation committees. For both of the ensemble methods, we employ weighted and unweighted voting to build different ensembles. Experimental results indicate that both bagging and cross-validation committees improve the word alignment results. The weighted ensembles perform much better than the unweighted ensembles accord- ing to our word alignment results. In addition, we analyze the effect of different sizes of training data on the bagging algorithm. …

Content truncated to 500 words