Locally Training the Log-Linear Model for SMT

Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 402–411, Jeju Island, Korea, 12–14 July 2012. c⃝2012 Association for Computational Linguistics Locally Training the Log-Linear Model for SMT Lemao Liu1, Hailong Cao1, Taro Watanabe2, Tiejun Zhao1, Mo Yu1, CongHui Zhu1 1School of Computer Science and Technology Harbin Institute of Technology, Harbin, China 2National Institute of Information and Communication Technology 3-5 Hikari-dai, Seika-cho, Soraku-gun, Kyoto, Japan {lmliu,hailong,tjzhao,yumo,chzhu}@mtlab.hit.edu.cn taro.watanabe@nict.go.jp Abstract In statistical machine translation, minimum error rate training (MERT) is a standard method for tuning a single weight with regard to a given development data. However, due to the diversity and uneven distribution of source sentences, there are two problems suffered by this method. First, its performance is highly dependent on the choice of a development set, which may lead to an unstable performance for testing. Second, translations become in- consistent at the sentence level since tuning is performed globally on a document level. In this paper, we propose a novel local training method to address these two problems. Un- like a global training method, such as MERT, in which a single weight is learned and used for all the input sentences, we perform training and testing in one step by learning a sentence- wise weight for each input sentence. We pro- pose efficient incremental training methods to put the local training into practice. In NIST Chinese-to-English translation tasks, our lo- cal training method significantly outperforms MERT with the maximal improvements up to 2.0 BLEU points, meanwhile its efficiency is comparable to that of the global method. 1 Introduction Och and Ney (2002) introduced the log-linear model for statistical machine translation (SMT), in which translation is considered as the following optimiza- tion problem: ˆe(f; W) = arg max e P(e|f; W) = arg max e exp � W · h(f, e) � � e′ exp � W · h(f, e′) � = arg max e � W · h(f, e) � , (1) where f and e (e′) are source and target sentences, respectively. h is a feature vector which is scaled by a weight W. Parameter estimation is one of the most important components in SMT, and var- ious training methods have been proposed to tune W. Some methods are based on likelihood (Och and Ney, 2002; Blunsom et al., 2008), error rate (Och, 2003; Zhao and Chen, 2009; Pauls et al., 2009; Gal- ley and Quirk, 2011), margin (Watanabe et al., 2007; Chiang et al., 2008) and ranking (Hopkins and May, 2011), and among which minimum error rate train- ing (MERT) (Och, 2003) is the most popular one. All these training methods follow the same pipeline: they train only a single weight on a given development set, and then use it to translate all the sentences in a test set. We call them a global train- ing method. One of its advantages is that it allows us to train a single weight offline and thereby it is effi- cient. However, …

Content truncated to 500 words