Using Word-Dependent Transition Models in HMM-Based Word Alignment for Statistical Machine Translation

Proceedings of the Second Workshop on Statistical Machine Translation, pages 80–87, Prague, June 2007. c⃝2007 Association for Computational Linguistics Using Word Dependent Transition Models in HMM based Word Alignment for Statistical Machine Translation Xiaodong He Microsoft Research One Microsoft Way Redmond, WA 98052 USA xiaohe@microsoft.com Abstract In this paper, we present a Bayesian Learn- ing based method to train word dependent transition models for HMM based word alignment. We present word alignment re- sults on the Canadian Hansards corpus as compared to the conventional HMM and IBM model 4. We show that this method gives consistent and significant alignment error rate (AER) reduction. We also con- ducted machine translation (MT) experi- ments on the Europarl corpus. MT results show that word alignment based on this method can be used in a phrase-based ma- chine translation system to yield up to 1% absolute improvement in BLEU score, compared to a conventional HMM, and 0.8% compared to a IBM model 4 based word alignment. 1 Introduction Word alignment is an important step of most modern approaches to statistical machine translation (Koehn et al., 2003). The classical approaches to word alignment are based on IBM models 1-5 (Brown et al., 1994) and the HMM based alignment model (Vogel et al., 1996) (Och and Ney, 2000a, 2000b), while recently discriminative approaches (Moore, 2006) and syntax based approaches (Zhang and Gildea, 2005) for word alignment are also studied. In this paper, we present improvements to the HMM based alignment model originally proposed by (Vogel et al., 1996, Och and Ney, 2000a). Although HMM based word alignment ap- proaches give good performance, one weakness of it is the coarse transition models. In the HMM based alignment model (Vogel et al., 1996), it is assumed that the HMM transition probabilities de- pend only on the jump width from the last state to the next state. Therefore, the knowledge of transi- tion probabilities given a particular source word e is not sufficiently modeled. In order to improve transition models in the HMM based alignment, Och and Ney (2000a) ex- tended the transition models to be word-class de- pendent. In that approach, words of the source lan- guage are first clustered into a number of word classes, and then a set of transition parameters is estimated for each word class. In (2002), Toutano- va et al. modeled self-transition (i.e., jump width is zero) probability separately from other transition probabilities. A word dependent self-transition model P(stay|e) is introduced to decide whether to stay at the current source word e at the next step, or jump to a different word. It was also shown that with the assumption that a source word with fertili- ty greater than one generates consecutive words in the target language, this probability approximates fertility modeling. Deng and Byrne in (2005) im- proved this idea. They proposed a word-to-phrase HMM in which a source word dependent phrase length model is used to model the approximate fertility, i.e., the length of consecutive target words generated by the source …

Content truncated to 500 words