Translation Model Generalization using Probability Averaging for Machine Translation

Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 304–312, Beijing, August 2010 Translation Model Generalization using Probability Averaging for Machine Translation Nan Duan1, Hong Sun School of Computer Science and Technology Tianjin University v-naduan@microsoft.com v-hongsun@microsoft.com Ming Zhou Microsoft Research Asia mingzhou@microsoft.com Abstract Previous methods on improving transla- tion quality by employing multiple SMT models usually carry out as a second- pass decision procedure on hypotheses from multiple systems using extra fea- tures instead of using features in existing models in more depth. In this paper, we propose translation model generalization (TMG), an approach that updates proba- bility feature values for the translation model being used based on the model it- self and a set of auxiliary models, aiming to enhance translation quality in the first- pass decoding. We validate our approach on translation models based on auxiliary models built by two different ways. We also introduce novel probability variance features into the log-linear models for further improvements. We conclude that our approach can be developed indepen- dently and integrated into current SMT pipeline directly. We demonstrate BLEU improvements on the NIST Chinese-to- English MT tasks for single-system de- codings, a system combination approach and a model combination approach.1 1 Introduction Current research on Statistical Machine Transla- tion (SMT) has made rapid progress in recent decades. Although differed on paradigms, such as phrase-based (Koehn, 2004; Och and Ney, 2004), hierarchical phrase-based (Chiang, 2007) and syntax-based (Galley et al., 2006; Shen et al., 2008; Huang, 2008), most SMT systems fol- 1 This work has been done while the author was visiting Microsoft Research Asia. low the similar pipeline and share common translation probability features which constitute the principal components of translation models. However, due to different model structures or data distributions, these features are usually as- signed with different values in different transla- tion models and result in translation outputs with individual advantages and shortcomings. In order to obtain further improvements, many approaches have been explored over multiple systems: system combination based on confu- sion network (Matusov et al., 2006; Rosti et al., 2007; Li et al., 2009a) develop on multiple N- best outputs and outperform primary SMT sys- tems; consensus-based methods (Li et al., 2009b; DeNero et al., 2010), on the other hand, avoid the alignment problem between translations can- didates and utilize n-gram consensus, aiming to optimize special decoding objectives for hypo- thesis selection. All these approaches act as the second-pass decision procedure on hypotheses from multiple systems by using extra features. They begin to work only after the generation of translation hypotheses has been finished. In this paper, we propose translation model generalization (TMG), an approach that takes effect during the first-pass decoding procedure by updating translation probability features for the translation model being used based on the model itself and a set of auxiliary models. Baye- sian Model Averaging is used to integrate values of identical features between models. Our con- tributions mainly include the following 3 aspects:  Alleviate the model bias problem …

Content truncated to 500 words