Dynamic Model Interpolation for Statistical Machine Translation

Proceedings of the Third Workshop on Statistical Machine Translation, pages 208–215, Columbus, Ohio, USA, June 2008. c⃝2008 Association for Computational Linguistics Abstract This paper presents a technique for class- dependent decoding for statistical machine translation (SMT). The approach differs from previous methods of class-dependent transla- tion in that the class-dependent forms of all models are integrated directly into the decod- ing process. We employ probabilistic mixture weights between models that can change dy- namically on a segment-by-segment basis depending on the characteristics of the source segment. The effectiveness of this approach is demonstrated by evaluating its performance on travel conversation data. We used the ap- proach to tackle the translation of questions and declarative sentences using class- dependent models. To achieve this, our system integrated two sets of models specifically built to deal with sentences that fall into one of two classes of dialog sentence: questions and dec- larations, with a third set of models built to handle the general class. The technique was thoroughly evaluated on data from 17 lan- guage pairs using 6 machine translation evaluation metrics. We found the results were corpus-dependent, but in most cases our sys- tem was able to improve translation perform- ance, and for some languages the improve- ments were substantial. 1 Introduction Topic-dependent modeling has proven to be an effective way to improve quality the quality of models in speech recognition (Iyer and Osendorf, 1994; Carter, 1994). Recently, experiments in the field of machine translation (Hasan and Ney, 2005; Yamamoto and Sumita, 2007; Finch et al. 2007, Foster and Kuhn, 2007) have shown that class- specific models are also useful for translation. In the method proposed by Yamamoto and Su- mita (2007), topic dependency was implemented by partitioning the data into sets before the decod- ing process commenced, and subsequently decod- ing these sets independently using different models that were specific to the class predicted for the source sentence by a classifier that was run over the source sentences in a pre-processing pass. Our approach is in many ways a generalization of this work. Our technique allows the use of multiple- model sets within the decoding process itself. The contributions of each model set can be controlled dynamically during the decoding through a set of interpolation weights. These weights can be changed on a sentence-by-sentence basis. The pre- vious approach is, in essence, the case where the interpolation weights are either 1 (indicating that the source sentence is the same topic as the model) or 0 (the source sentence is a different topic). One advantage of our proposed technique is that it is a soft approach. That is, the source sentence can be- long to multiple classes to varying degrees. In this respect our approach is similar to that of Foster and Kuhn (2007), however we used a probabilistic classifier to determine a vector of probabilities rep- resenting class-membership, rather than distance- based weights. These probabilities were used di- rectly as the mixture weights for the respective models in an interpolated model-set. …

Content truncated to 500 words