Mixtures of IBM Model 2

Mixtures of IBM Model 2∗ Jorge Civera, Alfons Juan Departamento de Sistemas Inform´aticos y Computaci´on. Universitat Polit`ecnica de Val`encia (Spain) {jcivera,ajuan}@dsic.upv.es Abstract Mixture modelling is a standard pattern classification technique. However, in statistical machine translation, the use of mixture modelling is still un- explored. Two main advantages of the mixture approach are first, its flex- ibility to find an appropriate tradeoff between model complexity and the amount of training data available and second, its capability to learn specific probability distributions that better fit subsets of the training dataset. This latter advantage is even more important in statistical machine translation, since it is well known that most of the current translation models proposed have limited application to restricted semantic domains. In this paper, we describe a mixture extension of the IBM model 2 along with the maximum likelihood estimation of its parameters through the EM algorithm and a dynamic-programming decoding algorithm for this mixture model. Prelim- inary experiments carried out on the Tourist task show that the mixture extension conveys a decrease in word-error rate of up to 15%. 1 Introduction Mixture modelling is a popular approach for density estimation in many scientific ar- eas (McLachlan & Peel, 2000). On the one hand, mixtures are flexible enough for find- ing an appropriate tradeoff between model complexity and the amount of training data available. Usually, model complexity is con- trolled by varying the number of mixture components while keeping the same para- metric form for all components. On the other hand, maximum likelihood estimation of mixture parameters can be reliably ac- complished by the well-known Expectation- Maximisation (EM) algorithm. The application of mixture modelling to translation models is still an unexplored field. One of the most interesting proper- ties of mixture modelling is its capability to learn a specific probability distribution in a multimodal dataset that better explains the general data generation process. In machine ∗This work has been supported by the Spanish project TIC2003-08681-C02 and the Ministerio de Educaci´on. y Ciencia. translation, these multimodal datasets are not an exception, but the general case. In- deed, it is easy to find corpora from which several topics could be drawn. These top- ics usually define sets of topic-specific lexi- cons that need to be translated taking into the sematic context in which they are found. This semantic ambiguity problem could be overcome by learning topic-dependent trans- lation models that capture together the se- mantic context and the translation process. In this paper, we describe a mixture ex- tension of the IBM model 2 (Brown et al., 1990) along with ts corresponding EM pa- rameter estimation. Furthermore, it was necessary to develop a specific search al- gorithm that finds the best translation for a given source sentence as defined by the IBM2 mixture model. A dynamic- programming search algorithm was imple- mented for this purpose as a mixture ex- tension of that presented in (Garc´ıa-Varea, Casacuberta, & Ney, 1998; Garc´ıa-Varea & Casacuberta, 2001). This paper is structured as follows. Next section reviews the IBM …

Content truncated to 500 words