Improving Word Alignment by Semi-Supervised Ensemble

Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 135–143, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics Improving Word Alignment by Semi-supervised Ensemble Shujian Huang1, Kangxi Li2, Xinyu Dai1, Jiajun Chen1 1State Key Laboratory for Novel Software Technology at Nanjing University Nanjing 210093, P.R.China {huangsj,daixy,chenjj}@nlp.nju.edu.cn 2School of Foreign Studies, Nanjing University Nanjing 210093, P.R.China richardlkx@126.com Abstract Supervised learning has been recently used to improve the performance of word alignment. However, due to the limited amount of labeled data, the performance of ”pure” supervised learning, which only used labeled data, is limited. As a re- sult, many existing methods employ fea- tures learnt from a large amount of unla- beled data to assist the task. In this pa- per, we propose a semi-supervised ensem- ble method to better incorporate both la- beled and unlabeled data during learning. Firstly, we employ an ensemble learning framework, which effectively uses align- ment results from different unsupervised alignment models. We then propose to use a semi-supervised learning method, namely Tri-training, to train classifiers us- ing both labeled and unlabeled data col- laboratively and further improve the result. Experimental results show that our meth- ods can substantially improve the quality of word alignment. The final translation quality of a phrase-based translation sys- tem is slightly improved, as well. 1 Introduction Word alignment is the process of learning bilin- gual word correspondences. Conventional word alignment process is treated as an unsupervised learning task, which automatically learns the cor- respondences between bilingual words using an EM style algorithm (Brown et al., 1993; Vogel et al., 1996; Och and Ney, 2003). Recently, su- pervised learning methods have been used to im- prove the performance. They firstly re-formalize word alignment as some kind of classification task. Then the labeled data is used to train the classification model, which is finally used to clas- sify unseen test data (Liu et al., 2005; Taskar et al., 2005; Moore, 2005; Cherry and Lin, 2006; Haghighi et al., 2009). It is well understood that the performance of supervised learning relies heavily on the fea- ture set. As more and more features are added into the model, more data is needed for train- ing. However, due to the expensive cost of la- beling, we usually cannot get as much labeled word alignment data as we want. This may limit the performance of supervised methods (Wu et al., 2006). One possible alternative is to use features learnt in some unsupervised manner to help the task. For example, Moore (2005) uses statistics like log-likelihood-ratio and conditional- likelihood-probability to measure word associa- tions; Liu et al. (2005) and Taskar et al. (2005) use results from IBM Model 3 and Model 4, re- spectively. Ayan and Dorr (2006) propose another way of incorporating unlabeled data. They first train some existing alignment models, e.g. IBM Model4 and Hidden Markov Model, using unlabeled data. The results of these models are then combined using a maximum entropy classifier, which is trained us- ing labeled data. This method …

Content truncated to 500 words