Multi-Rate HMMs for Word Alignment

Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 494–502, Sofia, Bulgaria, August 8-9, 2013 c⃝2013 Association for Computational Linguistics Multi-rate HMMs for Word Alignment Elif Eyig¨oz Computer Science University of Rochester Rochester, NY 14627 Daniel Gildea Computer Science University of Rochester Rochester, NY 14627 Kemal Oflazer Computer Science Carnegie Mellon University PO Box 24866, Doha, Qatar Abstract We apply multi-rate HMMs, a tree struc- tured HMM model, to the word-alignment problem. Multi-rate HMMs allow us to model reordering at both the morpheme level and the word level in a hierarchical fashion. This approach leads to better ma- chine translation results than a morpheme- aware model that does not explicitly model morpheme reordering. 1 Introduction We present an HMM-based word-alignment model that addresses transitions between mor- pheme positions and word positions simultane- ously. Our model is an instance of a multi-scale HMM, a widely used method for modeling dif- ferent levels of a hierarchical stochastic process. In multi-scale modeling of language, the deepest level of the hierarchy may consist of the phoneme sequence, and going up in the hierarchy, the next level may consist of the syllable sequence, and then the word sequence, the phrase sequence, and so on. By the same token, in the hierarchical word- alignment model we present here, the lower level consists of the morpheme sequence and the higher level the word sequence. Multi-scale HMMs have a natural application in language processing due to the hierarchical nature of linguistic structures. They have been used for modeling text and handwriting (Fine et al., 1998), in signal processing (Willsky, 2002), knowledge extraction (Skounakis et al., 2003), as well as in other fields of AI such as vision (Li et al., 2006; Luettgen et al., 1993) and robotics (Theocharous et al., 2001). The model we propose here is most similar to multi-rate HMMs (C¸ etin et al., 2007), which were applied to a classification problem in industrial machine tool wear. The vast majority of languages exhibit morphol- ogy to some extent, leading to various efforts in machine translation research to include morphol- ogy in translation models (Al-Onaizan et al., 1999; Niessen and Ney, 2000; ˇCmejrek et al., 2003; Lee, 2004; Chung and Gildea, 2009; Yeniterzi and Oflazer, 2010). For the word-alignment problem, Goldwater and McClosky (2005) and Eyig¨oz et al. (2013) suggested word alignment models that ad- dress morphology directly. Eyig¨oz et al. (2013) introduced two-level align- ment models (TAM), which adopt a hierarchi- cal representation of alignment: the first level in- volves word alignment, the second level involves morpheme alignment. TAMs jointly induce word and morpheme alignments using an EM algorithm. TAMs can align rarely occurring words through their frequently occurring morphemes. In other words, they use morpheme probabilities to smooth rare word probabilities. Eyig¨oz et al. (2013) introduced TAM 1, which is analogous to IBM Model 1, in that the first level is a bag of words in a pair of sentences, and the sec- ond level is a bag of morphemes. By introducing distortion probabilities …

Content truncated to 500 words