Search-Aware Tuning for Hierarchical Phrase-based Decoding

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1282–1291, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Search-Aware Tuning for Hierarchical Phrase-based Decoding Feifei Zhai Queens College City University of New York ffzhai2012@gmail.com Liang Huang Kai Zhao Queens College & Graduate Center City University of New York {lianghuang.sh,kzhao.hf}@gmail.com Abstract Parameter tuning is a key problem for sta- tistical machine translation (SMT). Most popular parameter tuning algorithms for SMT are agnostic of decoding, result- ing in parameters vulnerable to search er- rors in decoding. The recent research of “search-aware tuning” (Liu and Huang, 2014) addresses this problem by consid- ering the partial derivations in every de- coding step so that the promising ones are more likely to survive the inexact de- coding beam. We extend this approach from phrase-based translation to syntax- based translation by generalizing the eval- uation metrics for partial translations to handle tree-structured derivations in a way inspired by inside-outside algorithm. Our approach is simple to use and can be ap- plied to most of the conventional parame- ter tuning methods as a plugin. Extensive experiments on Chinese-to-English trans- lation show significant BLEU improve- ments on MERT, MIRA and PRO. 1 Introduction Efforts in parameter tuning algorithms for SMT, such as MERT (Och, 2003; Galley et al., 2013), MIRA (Watanabe et al., 2007; Chiang et al., 2009; Cherry and Foster, 2012; Chiang, 2012), and PRO (Hopkins and May, 2011) have improved the translation quality considerably in the past decade. These tuning algorithms share the same characteristic that they treat the decoder as a black box. This decoding insensitiveness has two ef- fects: 1) the parameter tuning algorithm can be more general to choose the most effective decod- ing paradigm for different language pairs; 2) how- ever, it also means that the learned parameters may not fit the decoding algorithm well, so that promis- ing partial translations might be pruned out due to the beam search decoding. Recent researches reveal that the parameter tun- ing algorithms tailored for specific decoding al- gorithms can be beneficial in general structured prediction problems (Huang et al., 2012), and in machine translation (Yu et al., 2013; Zhao et al., 2014; Liu and Huang, 2014). Particularly, Liu and Huang (2014) show that by requiring the conven- tional parameter tuning algorithms to consider the final decoding results (full translations) as well as the intermediate decoding states (partial transla- tions) at the same time, the inexact decoding can be significantly improved so that correct interme- diate partial translations are more likely to survive the beam. However, the underlying phrase-based decoding model suffers from limited distortion, and thus, may not be flexible enough for translat- ing language pairs that are syntactically different, which require long distance reordering. In order to better handle long distance reorder- ing which beyonds the capability of phrase-based MT, we extend the search-aware tuning frame- work from phrase-based MT to syntax-based MT, in particular the hierarchical phrase-based transla- tion model (HIERO) (Chiang, 2007). One key advantage …

Content truncated to 500 words