Alignment by Agreement

Proceedings of the Human Language Technology Conference of the North American Chapter of the ACL, pages 104–111, New York, June 2006. c⃝2006 Association for Computational Linguistics Alignment by Agreement Percy Liang UC Berkeley Berkeley, CA 94720 pliang@cs.berkeley.edu Ben Taskar UC Berkeley Berkeley, CA 94720 taskar@cs.berkeley.edu Dan Klein UC Berkeley Berkeley, CA 94720 klein@cs.berkeley.edu Abstract We present an unsupervised approach to symmetric word alignment in which two simple asymmetric models are trained jointly to maximize a combination of data likelihood and agreement between the models. Compared to the stan- dard practice of intersecting predictions of independently-trained models, joint train- ing provides a 32% reduction in AER. Moreover, a simple and efficient pair of HMM aligners provides a 29% reduction in AER over symmetrized IBM model 4 predictions. 1 Introduction Word alignment is an important component of a complete statistical machine translation pipeline (Koehn et al., 2003). The classic approaches to un- supervised word alignment are based on IBM mod- els 1–5 (Brown et al., 1994) and the HMM model (Ney and Vogel, 1996) (see Och and Ney (2003) for a systematic comparison). One can classify these six models into two groups: sequence-based models (models 1, 2, and HMM) and fertility-based models (models 3, 4, and 5).1 Whereas the sequence-based models are tractable and easily implemented, the more accurate fertility-based models are intractable and thus require approximation methods which are 1IBM models 1 and 2 are considered sequence-based models because they are special cases of HMMs with transitions that do not depend on previous states. difficult to implement. As a result, many practition- ers use the complex GIZA++ software package (Och and Ney, 2003) as a black box, selecting model 4 as a good compromise between alignment quality and efficiency. Even though the fertility-based models are more accurate, there are several reasons to consider av- enues for improvement based on the simpler and faster sequence-based models. First, even with the highly optimized implementations in GIZA++, models 3 and above are still very slow to train. Sec- ond, we seem to have hit a point of diminishing re- turns with extensions to the fertility-based models. For example, gains from the new model 6 of Och and Ney (2003) are modest. When models are too complex to reimplement, the barrier to improvement is raised even higher. Finally, the fertility-based models are asymmetric, and symmetrization is com- monly employed to improve alignment quality by intersecting alignments induced in each translation direction. It is therefore natural to explore models which are designed from the start with symmetry in mind. In this paper, we introduce a new method for word alignment that addresses the three issues above. Our development is motivated by the observation that in- tersecting the predictions of two directional models outperforms each model alone. Viewing intersec- tion as a way of finding predictions that both models agree on, we take the agreement idea one step fur- ther. The central idea of our approach is to not only make the predictions of the models agree …

Content truncated to 500 words