A Deterministic Annealing-Based Training Algorithm For Statistical Machine Translation Models

A Deterministic Annealing-Based Training Algorithm For Statistical Machine Translation Models Pascual Mart´ınez G´omez1, Kei Hashimoto2, Yoshihiko Nankaku2, Keiichi Tokuda2, Germ´an Sanchis-Trilles1 (1) Instituto Tecnol´ogico de Inform´atica Universidad Polit´ecnica de Valencia {pmartinez,gsanchis}@dsic.upv.es (2) Tokuda & Lee Laboratory Nagoya Institute of Technology {bonanza,nankaku,tokuda}@sp.nitech.ac.jp Abstract This paper proposes the use of a Determin- istic Annealing Expectation-Maximization (DAEM) algorithm to estimate the word- alignments involved in the statistical trans- lation process. This approach is aimed to overcome the problem of the local maxima in complex alignment models, thus making unnecessary to iterate with previous and simpler ones. Using the DAEM algorithm allows us to explore the power of highly expressive statistical alignment models without the experimental limitations of working with non-convex models, while, at the same time, observing consistent improvements in translation quality. Experimental results show that, by us- ing an appropriate temperature scheduling, equal or better estimations are obtained in- dependently of the initial parameter esti- mates. 1 Introduction Statistical Machine Translation (SMT) systems use mathematical models to describe the translation task and to estimate the probabilities involved in the process. Brown et al. (1993) established the SMT grounds formulating the probability of translating a source sentence f into a target sentence e, as ˆe = argmax e Pr(f | e) · Pr(e) (1) where Pr(e) stands for the language model and Pr(f|e) is the translation model. The language c⃝ 2010 European Association for Machine Translation. model is usually based on n-grams, accounting mainly for the word-order with the purpose of avoiding ill-formed sentence e, and the translation model accounts for the probability of the words of e being a good translation of the words of f. In word-to-word SMT, word-alignments were used to model the distribution Pr(f | e). In this context, a hidden variable a is introduced to repre- sent word-correspondences in a bilingual sentence pair. Introducing the alignment concept: Pr(f | e) = � a Pr(f, a | e) (2) In practice, however, the direct modelling of the posterior probability Pr(f | e) has been widely adopted. To this purpose, different authors (Pap- ineni et al., 1998; Och and Ney, 2002) propose the use of the so-called log-linear models, where the decision rule is given by the expression ˆe = argmax e M � m=1 λmhm(f, e) (3) where hm(x, y) is a score function representing an important feature for the translation of f into e, M is the number of models (or features) and λm are the weights of the log-linear combination. In order to introduce context information, mod- ern state-of-the-art SMT systems no longer im- plement a word-by-word translation model as de- scribed by Equation 2, but rather segment-to- segment. Phrase-Based (PB) SMT systems, which constitute the most popular implementation of log- linear SMT models, still rely heavily on word- alignment models in order to obtain these seg- ments. Once the word alignments have been ob- tained, all bilingual phrases (i.e. subsequent word segments) coherent with the word alignment are [EAMT May 2010 St Raphael, France] extracted. Hence, …

Content truncated to 500 words