Incorporating Position Information into a Maximum Entropy/Minimum Divergence Translation Model

In: Proceedings of CoNLL-2000 and LLL-2000, pages 37-42, Lisbon, Portugal, 2000. Incorporating Position Information into a Maximum Entropy/Minimum Divergence Translation Model George Foster RALI, Universit6 de Montr6al foster@iro, umontreal, ca Abstract I describe two methods for incorporating infor- mation about the relative positions of bilingual word pairs into a Maximum Entropy/Minimum Divergence translation model. The better of the two achieves over 40% lower test corpus perplex- ity than an equivalent combination of a trigram language model and the classical IBM transla- tion model 2. 1 Introduction Statistical Machine Translation (SMT) systems use a model ofp(tls), the probability that a text s in the source language will translate into a text t in the target language, to determine the best translation for a given source text. A straight- forward way of modeling this distribution is to apply a chain-rule expansion of the form: ItL p(tlS ) = Hp(tiltl...ti-l,S), i=1 (i) where ti denotes the ith token in t. 1 The objects to be modeled in this case belong to the family of conditional distributions p(wlhi, s), the prob- ability of the ith word in t, given the tokens which precede it and the source text. The main motivation for modeling p(tls ) in terms of p(wlhi , s) is that it simplifies the "de- coding" problem of finding the most likely tar- get text. In particular, if hi is known, finding the best word at the current position requires only a straightforward search through the target 1This ignores the issue of normalization over target texts of all possible lengths, which can be easily enforced when desired by using a stop token or a prior distribution over lengths. vocabulary, and efficient dynamic-programming based heuristics can be used to extend this to sequences of words. This is very important for applications such as TransType (Foster et al., 1997; Langlais et al., 2000), where the task is to make real-time predictions of the text a hu- man translator will type next, based on the source text under translation and some prefix of the target text that has already been typed. The standard "noisy channel" approach used in SMT, where p(tls ) c< p(t)p(slt), is generally too expensive for such applications because it does not permit direct calculation of the probabil- ity of a word or sequence of words beginning at the current position. Complex and expensive search strategies are required to find the best target text in this approach (Garcfa-Varea et al., 1998; Niessen et al., 1998; Ochet al., 1999; Wang and Waibel, 1998). The challenge in modeling p(wlhi,s ) is to combine two disparate sources of conditioning information in an effective way. One obvious strategy is to use a linear combination of sep- arate language and translation components, of the form: p(w[hi, s) -- Ap(w[hi) + (1 - A)p(w[i, s). (2) where p(w[hi) is a language model, p(wli , s) is a translation model, and A E [0, 1] is a com- bining weight. However, this appears to be a weak technique (Langlais …

Content truncated to 500 words