Adaptive Development Data Selection for Log-linear Model in Statistical Machine Translation

Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 662–670, Beijing, August 2010 Adaptive Development Data Selection for Log-linear Model in Statistical Machine Translation Mu Li Microsoft Research Asia muli@microsoft.com Yinggong Zhao∗ Nanjing University zhaoyg@nlp.nju.edu.cn Dongdong Zhang Microsoft Research Asia dozhang@microsoft.com Ming Zhou Microsoft Research Asia mingzhou@microsoft.com Abstract This paper addresses the problem of dy- namic model parameter selection for log- linear model based statistical machine translation (SMT) systems. In this work, we propose a principled method for this task by transforming it to a test data de- pendent development set selection prob- lem. We present two algorithms for au- tomatic development set construction, and evaluated our method on several NIST data sets for the Chinese-English trans- lation task. Experimental results show that our method can effectively adapt log-linear model parameters to different test data, and consistently achieves good translation performance compared with conventional methods that use a fixed model parameter setting across different data sets. 1 Introduction In recent years, log-linear model (Och and Ney, 2002) has been a mainstream method to formu- late statistical models for machine translation. Us- ing this formulation, various kinds of relevant properties and data statistics used in the transla- tion process, either on the monolingual-side or on the bilingual-side, are encoded and used as real- valued feature functions, thus it provides an ef- fective mathematical framework to accommodate a large variety of SMT formalisms with different computational linguistic motivations. ∗This work was done while the author was visiting Mi- crosoft Research Asia. Formally, in a log-linear SMT model, given a source sentence f, we are to find a translation e∗ with largest posterior probability among all possi- ble translations: e∗ = argmax e Pr(e|f) and the posterior probability distribution Pr(e|f) is directly approximated by a log-linear formula- tion: Pr(e|f) = pλ(e|f) = exp(�M m=1 λmhm(e, f)) � e′ exp(�M m=1 λmhm(e′, f)) (1) in which hm’s are feature functions and λ = (λ1, . . . , λM) are model parameters (feature weights). For a successful practical log-linear SMT model, it is usually a combined result of the sev- eral efforts: • Construction of well-motivated SMT models • Accurate estimation of feature functions • Appropriate scaling of log-linear model fea- tures (feature weight tuning). In this paper, we focus on the last mentioned issue – parameter tuning for log-linear model. In general, log-linear model parameters are opti- mized on a held-out development data set. Us- ing this method, similarly to many machine learn- ing tasks, the model parameters are solely tuned based on the development data, and the optimal- ity of obtained model on unseen test data relies on the assumption that both development and test data observe identical probabilistic distribution, 662 which often does not hold for real-world data. The goal of this paper is to investigate novel meth- ods for test data dependent model parameter se- lection. We begin with discussing the principle of parameter learning for log-linear SMT models, and explain the rationale of task transformation from …

Content truncated to 500 words