Extensions to HMM-based Statistical Word Alignment Models

Extensions to HMM-based Statistical Word Alignment Models Kristina Toutanova, H. Tolga Ilhan and Christopher D. Manning Department of Computer Science Stanford University Stanford, CA 94305-9040 USA kristina@cs.stanford.edu ilhan@stanford.edu manning@cs.stanford.edu Abstract This paper describes improved HMM-based word level alignment models for statistical machine translation. We present a method for using part of speech tag information to improve alignment accu- racy, and an approach to modeling fertility and cor- respondence to the empty word in an HMM align- ment model. We present accuracy results from eval- uating Viterbi alignments against human-judged alignments on the Canadian Hansards corpus, as compared to a bigram HMM, and IBM model 4. The results show up to 16% alignment error reduc- tion. 1 Introduction The main task in statistical machine translation is to model the string translation probability ���� �� � where the string � in one language is translated into another language as string �� � . We refer to �� � as the source language string and � as the target language string in accordance with the noisy chan- nel terminology used in the IBM models of (Brown et al., 1993). Word-level translation models assume a pairwise mapping between the words of the source and target strings. This mapping is generated by alignment models. In this paper we present exten- sions to the HMM alignment model of (Vogel et al., 1996; Och and Ney, 2000b). Some of our extensions are applicable to other alignment models as well and are of general utility.1 For most language pairs huge amounts of parallel corpora are not readily available whereas monolin- gual resources such as taggers are more often avail- able. Little research has gone into exploring the po- 1This paper was supported in part by the National Science Foundation under Grants IIS-0085896 and IIS-9982226. The authors would also like to thank the various reviewers for their helpful comments on earlier versions. tential of part of speech information to better model translation probabilities and permutation probabili- ties. Melamed (2000) uses a very broad classifica- tion of words (content, function and several punctu- ation classes) to estimate class-specific parameters for translation models. Fung and Wu (1995) adapt English tags for Chinese language modeling using Coerced Markov Models. They use English POS classes as states of the Markov Model to generate Chinese language words. In this paper we use POS tag information to incorporate prior knowledge of word translation and to model local word order vari- ation. We show that using this information can help in the translation modeling task. Many alignment models assume a one to many mapping from source language words to target lan- guage words, such as the IBM models 1-5 of Brown et al. (1993) and the HMM alignment model of (Vo- gel et al., 1996). In addition, the IBM Models 3, 4 and 5 include a fertility model  �� � where  is the number of words aligned to a source word � . In HMM-based alignment word fertilities are not mod- eled. The alignment …

Content truncated to 500 words