Approach to Selecting Best Development Set for Phrase-Based Statistical Machine Translation

Approach to Selecting Best Development Set for Phrase-Based Statistical Machine Translation∗∗∗∗ Peng Liu, Yu Zhou, and Chengqing Zong National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences Room 1010, 95 Zhongguancun East Road, Beijing 100190, China {pliu, yzhou, cqzong}@nlpr.ia.ac.cn Abstract. In phrase-based statistical machine translation system, the parameters of model are usually obtained from minimum error rate training (MERT) on the development set. So the development set has a great influence on the performance of the translation system. Generally, more development set will achieve more effective and robust parameters but consume much more time on MERT process. In this paper, we propose two methods to select sentences from the large development set, based on the phrase and the sentence structure respectively. The experimental results show that our methods can get better translation performance than the baseline system on the compact development set by using a state-of-the-art SMT system. Keywords: Development set selecting, Domain adaption, Phrase-based statistical machine translation, Similarity measure ∗ The research work has been partially funded by the Natural Science Foundation of China under grant No.60736014, 60723005 and 90820303, the National Key Technology R&D Program under grant No. 2006BAH03B02, the Hi-Tech Research and Development Program (863 Program) of China under grant No. 2006AA010108-4, and also supported by the China-Singapore Institute of Digital Media as well. Copyright 2009 by Peng Liu, Yu Zhou, and Chengqing Zong 1 Introduction In recent years, the phrase-based statistical machine translation model obtains more attention and achieves good translation performance. But the quality of the translation results is greatly influenced by the model’s parameters. How to get a group of parameters which can make good translation results becomes a problem which we focus on. In general, we could get the optimized parameters though minimum error rate training (MERT) on the development set (Och, 2003). The MERT will continue running until the BLEU score on the development set convergences and then we translate the test set to the target language using this group of parameters. Since we get the optimized parameters on the development set, it becomes an important factor to the quality of the final translation results. Usually, we run the MERT on all of the development set, but when the development set is in large-scale and there are many long sentences included in it, the MERT will consume too long time on translation and parameters’ adjusting. Nevertheless we can not sure whether the parameters trained on it are optimal. So what we are interested in are: 1) How many sentences are adequate for the MERT and what kind of sentences contribute more to the MERT? 2) How can we select such development set to obtain more effective and robust parameters with less time and without performance losing? Based on the analysis above, our aim is to find a method to select sentences from the whole development set, and use the new development set we can save time on MERT process as well as improve the performance of the translation system. …

Content truncated to 500 words