Simpler Is Better: Re-evaluation of Default Word Alignment Models in Statistical MT

Simpler Is Better: Re-evaluation of Default Word Alignment Models in Statistical MT Mark Fishel Department of Computer Science, University of Tartu, Liivi 2 - 307, Tartu 50606, Estonia fishel@ut.ee Abstract. Although several recent studies have shown that alignment quality is a poor in- dicator of the resulting translation quality, the word alignment models currently considered to be default (the so-called IBM models and HMM-based alignment) have been evaluated using the alignment error rate. We argue that from a machine translation perspective it makes sense to use simpler alignment models. Here we show that not only do the sequential models result in the same or better translation quality, but even from the set of sequential alignment models simpler ones can match the performance of the HMM-based model, whereas using computationally less expensive and faster algorithms to train and align new sentence pairs. Empirical evaluation is performed on a phrase-based and a parsing-based translation system. Keywords: Word alignment, machine translation, relative reordering 1 Introduction A majority of state-of-the-art statistical machine translation (SMT) systems operate with multi- word units but still use word alignment as an intermediate step for learning the translation mod- els. Such is the case for two wide-spread machine translation frameworks: phrase-based SMT of (Koehn et al., 2003) and hierarchical phrase-based SMT of (Chiang, 2005). In phrase-based sys- tems word alignment is used to construct phrase tables and in hierarchical phrase-based systems – to extract the synchronous grammar rules. The word alignment models that are currently considered as default are the so-called IBM models 1 to 5 (Brown et al., 1993) and the HMM-based alignment model (Vogel et al., 1996). The main work evaluating them is (Och and Ney, 2003) where they are compared in the context of word alignment only (i.e. based on the alignment error rate). Specifically, the default setup of the well known implementation of the models, GIZA++, is derived from (Och and Ney, 2003) and involves training the following models in sequence: IBM model 1, HMM-based model, IBM model 3 and IBM model 4. Although Och and Ney (2003) mention briefly that “improved alignment quality yields an improved subjective quality of the statistical machine translation system as well”, a number of recent studies suggest otherwise – namely, that the correlation between the alignment quality and the quality of the resulting translation is rather weak (see the following section on related work). This suggests that the best default word alignment models are not necessarily optimal in terms of the resulting translation quality. Some recent works (e.g. (Liang et al., 2006) or (DeNero and Klein, 2007)) are already based on the sequential HMM word alignment model, rather than the fertility-based models 3 or 4 . The former is computationally less complex and at the same time still includes the essential parts of the word alignment (i.e. lexical correspondence and changes in word order). Copyright 2010 by Mark Fishel PACLIC 24 Proceedings 381 In this work we show that a simpler alignment model introduced together with HMM-based alignment in …

Content truncated to 500 words