Transductive Minimum Error Rate Training for Statistical Machine Translation

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 641–648, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Transductive Minimum Error Rate Training for Statistical Machine Translation Yinggong Zhao1∗, Shujie Liu2, Yangsheng Ji1, Jiajun Chen1, Guodong Zhou3 1State Key Laboratory for Novel Software Technology at Nanjing University Nanjing 210093, China {zhaoyg,jiys,chenjj}@nlp.nju.edu.cn 2School of Computer Science and Technology, Harbin Institute of Technology Harbin, China shujieliu@mtlab.hit.edu.cn 3School of Computer Science and Technology, Soochow University Suzhou, China gdzhou@suda.edu.cn Abstract This paper investigates parameter adap- tation in Statistical Machine Transla- tion(SMT). To overcome the parameter bias-estimation problem with Minimum Error Rate Training(MERT), we extend it under a transductive learning framework, by iteratively re-estimating the parame- ters using both development and test da- ta, in which the translation hypotheses of the test data are used as pseudo ref- erences. Furthermore, in order to over- come the over-training and unstableness problems respectively in employing such pseudo references, a termination criterion using a hyper-parameter and a Minimum Bayes Risk(MBR)-based hypothesis se- lection method are proposed in our work. Experimental results show that the trans- ductive MERT method could yield signif- icant performance improvements over a strong baseline on a large-scale Chinese- to-English translation task. 1 Introduction Machine translation (MT) is the automatic trans- lation from one natural language into another us- ing computer, while SMT is an approach to MT that is characterized by the use of machine learn- ing methods (Lopez, 2008). Nowadays, SMT is usually built on a log-linear model (Och and Ney, 2002), which can be abstracted into two steps: the ∗Part of this work is done during the first author’s intern- ship at Microsoft Research Asia. first one is model training, i.e., learning features from large collection of bilingual parallel corpus; and the other is parameter estimation, in which the feature weight is tuned on an independent devel- opment dataset. More specifically, for each source sentence f, we search for its final translation e∗ among all pos- sible translations based on the following equation: P(e∗|f) = arg max e Pr(e|f) (1) Under the log-linear model, the posterior proba- bility Pr(e|f) can be decomposed as: Pr(e|f) = pλ(e|f) = exp(�M m=1(λm · hm(e, f))) � e′ exp(�M m=1(λm · hm(e′, f))) (2) where each hm(e, f) is a feature function and λm is the corresponding weight for m=1,...,M. In SMT, there are three sections of data: the training data for feature estimation, the develop- ment data for weight tuning, and the test data for final evaluation. However, these three parts may belong to different domains, leading to distribu- tion variations, which indicates that the features and weights learned from the data may be bias- estimated. As a result, the adaptation for features and their corresponding weights are both impor- tant issues in SMT. In this article we focus on the latter issue, i.e., the model parameter adaptation. From the view- point of machine learning, development data is la- beled data used for parameter learning, while test data is unlabeled …

Content truncated to 500 words