Constraining the Phrase-Based, Joint Probability Statistical Translation Model

Proceedings of the Workshop on Statistical Machine Translation, pages 154–157, New York City, June 2006. c⃝2006 Association for Computational Linguistics Constraining the Phrase-Based, Joint Probability Statistical Translation Model Alexandra Birch Chris Callison-Burch Miles Osborne Philipp Koehn School of Informatics University of Edinburgh 2 Buccleuch Place Edinburgh, EH8 9LW, UK a.c.birch-mayne@sms.ed.ac.uk Abstract The joint probability model proposed by Marcu and Wong (2002) provides a strong probabilistic framework for phrase-based statistical machine translation (SMT). The model’s usefulness is, however, limited by the computational complexity of estimat- ing parameters at the phrase level. We present the first model to use word align- ments for constraining the space of phrasal alignments searched during Expectation Maximization (EM) training. Constrain- ing the joint model improves performance, showing results that are very close to state- of-the-art phrase-based models. It also al- lows it to scale up to larger corpora and therefore be more widely applicable. 1 Introduction Machine translation is a hard problem because of the highly complex, irregular and diverse nature of natural languages. It is impossible to accurately model all the linguistic rules that shape the trans- lation process, and therefore a principled approach uses statistical methods to make optimal decisions given incomplete data. The original IBM Models (Brown et al., 1993) learn word-to-word alignment probabilities which makes it computationally feasible to estimate model parameters from large amounts of train- ing data. Phrase-based SMT models, such as the alignment template model (Och, 2003), improve on word-based models because phrases provide local context which leads to better lexical choice and more reliable local reordering. However, most phrase-based models extract their phrase pairs from previously word-aligned corpora using ad- hoc heuristics. These models perform no search for optimal phrasal alignments. Even though this is an efficient strategy, it is a departure from the rigorous statistical framework of the IBM Models. Marcu and Wong (2002) proposed the joint probability model which directly estimates the phrase translation probabilities from the corpus in a theoretically governed way. This model neither relies on potentially sub-optimal word alignments nor on heuristics for phrase extraction. Instead, it searches the phrasal alignment space, simultane- ously learning translation lexicons for both words and phrases. The joint model has been shown to outperform standard models on restricted data sets such as the small data track for Chinese-English in the 2004 NIST MT Evaluation (Przybocki, 2004). However, considering all possible phrases and all their possible alignments vastly increases the computational complexity of the joint model when compared to its word-based counterpart. In this paper, we propose a method of constraining the search space of the joint model to areas where most of the unpromising phrasal alignments are eliminated and yet as many potentially useful alignments as possible are still explored. The joint model is constrained to phrasal alignments which do not contradict a set high confidence word alignments for each sentence. These high con- fidence alignments could incorporate information from both statistical and linguistic sources. In this paper we use the points of high confidence from the …

Content truncated to 500 words