HMM Word and Phrase Alignment for Statistical Machine Translation

Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), pages 169–176, Vancouver, October 2005. c⃝2005 Association for Computational Linguistics HMM Word and Phrase Alignment for Statistical Machine Translation Yonggang Deng1 , William Byrne1,2 Center for Language and Speech Processing, Johns Hopkins University Baltimore, MD 21210, USA 1 Machine Intelligence Lab, Cambridge University Engineering Department Trumpington Street, Cambridge CB2 1PZ, UK 2 dengyg@jhu.edu , wjb31@cam.ac.uk Abstract HMM-based models are developed for the alignment of words and phrases in bitext. The models are formulated so that align- ment and parameter estimation can be per- formed efficiently. We find that Chinese- English word alignment performance is comparable to that of IBM Model-4 even over large training bitexts. Phrase pairs extracted from word alignments generated under the model can also be used for phrase-based translation, and in Chinese to English and Arabic to English transla- tion, performance is comparable to sys- tems based on Model-4 alignments. Di- rect phrase pair induction under the model is described and shown to improve trans- lation performance. 1 Introduction Describing word alignment is one of the fundamen- tal goals of Statistical Machine Translation (SMT). Alignment specifies how word order changes when a sentence is translated into another language, and given a sentence and its translation, alignment spec- ifies translation at the word level. It is straightfor- ward to extend word alignment to phrase alignment: two phrases align if their words align. Deriving phrase pairs from word alignments is now widely used in phrase-based SMT. Parameters of a statistical word alignment model are estimated from bitext, and the model is used to generate word alignments over the same bitext. Phrase pairs are ex- tracted from the aligned bitext and used in the SMT system. With this approach the quality of the under- lying word alignments can have a strong influence on phrase-based SMT system performance. The common practice therefore is to extract phrase pairs from the best attainable word alignments. Currently, Model-4 alignments (Brown and others, 1993) as produced by GIZA++ (Och and Ney, 2000) are often the best that can be obtained, especially with large bitexts. Despite its modeling power and widespread use, Model-4 has shortcomings. Its formulation is such that maximum likelihood parameter estimation and bitext alignment are implemented by approximate, hill-climbing, methods. Consequently parameter es- timation can be slow, memory intensive, and diffi- cult to parallelize. It is also difficult to compute statistics under Model-4. This limits its usefulness for modeling tasks other than the generation of word alignments. We describe an HMM alignment model devel- oped as an alternative to Model-4. In the word align- ment and phrase-based translation experiments to be presented, its performance is comparable or im- proved relative to Model-4. Practically, we can train the model by the Forward-Backward algorithm, and by parallelizing estimation, we can control memory usage, reduce the time needed for training, and in- crease the bitext used for training. We can also com- pute statistics under the model in ways not …

Content truncated to 500 words