Tuning SMT with a Large Number of Features via Online Feature Grouping

Tuning SMT with A Large Number of Features via Online Feature Grouping Lemao Liu1, Tiejun Zhao1, Taro Watanabe2, Eiichiro Sumita2 1School of Computer Science and Technology Harbin Institute of Technology, Harbin, China 2National Institute of Information and Communications Technology 3-5 Hikari-dai, Seika-cho, Soraku-gun, Kyoto, Japan {lmliu|tjzhao}@mtlab.hit.edu.cn {taro.watanabe|eiichiro.sumita}@nict.go.jp Abstract In this paper, we consider the tuning of sta- tistical machine translation (SMT) mod- els employing a large number of features. We argue that existing tuning methods for these models suffer serious sparsity prob- lems, in which features appearing in the tuning data may not appear in the test- ing data and thus those features may be over tuned in the tuning data. As a result, we face an over-fitting problem, which limits the generalization abilities of the learned models. Based on our analysis, we propose a novel method based on feature grouping via OSCAR to overcome these pitfalls. Our feature grouping is imple- mented within an online learning frame- work and thus it is efficient for a large scale (both for features and examples) of learning in our scenario. Experiment re- sults on IWSLT translation tasks show that the proposed method significantly outper- forms the state of the art tuning methods. 1 Introduction Since the introduction of log-linear based SMT (Och and Ney, 2002), tuning has been a hot topic. Various methods have been explored: their objec- tives are either error rates (Och, 2003), hinge loss (Watanabe et al., 2007; Chiang et al., 2008) or ranking loss (Hopkins and May, 2011), and they are either batch training or online training meth- ods. In this paper, we consider tuning translation models with a large number of features such as lexical, n-gram level and rule level features, where the number of features is largely greater than the number of bilingual sentences. Practically, exist- ing tuning methods such as PRO and MIRA might This joint work was done while the first author visited NICT. be applied in our scenario, however, they will suf- fer from some pitfalls as well, which have been less investigated in previous works. One of pitfalls is that these features are so sparse that many features which are potentially useful for a test set may not be included in a given tuning set, and many useless features for testing will be over tuned on the developement set meanwhile. As a result, the generalization abilities of features are limited due to the mismatch between the testing data and the tuning data, and over-fitting occurs. One practice is to tune translation models on a larger tuning set, such as the entire training data (Xiao et al., 2011; Simianer et al., 2012), in the hope that more features would be included during tuning. However, tuning robust weights for trans- lation models has additional requirements to a tun- ing set. Firstly, multiple reference translations in the tuning data are helpful for better tuning, es- pecially when testing data contains multiple refer- ence translations. Secondly, the closeness between the tuning set and a test set …

Content truncated to 500 words