Improved Reordering for Phrase-Based Translation using Sparse Features

Proceedings of NAACL-HLT 2013, pages 22–31, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics Improved Reordering for Phrase-Based Translation using Sparse Features Colin Cherry National Research Council Canada Colin.Cherry@nrc-cnrc.gc.ca Abstract There have been many recent investigations into methods to tune SMT systems using large numbers of sparse features. However, there have not been nearly so many examples of helpful sparse features, especially for phrase- based systems. We use sparse features to ad- dress reordering, which is often considered a weak point of phrase-based translation. Using a hierarchical reordering model as our base- line, we show that simple features coupling phrase orientation to frequent words or word- clusters can improve translation quality, with boosts of up to 1.2 BLEU points in Chinese- English and 1.8 in Arabic-English. We com- pare this solution to a more traditional max- imum entropy approach, where a probability model with similar features is trained on word- aligned bitext. We show that sparse decoder features outperform maximum entropy hand- ily, indicating that there are major advantages to optimizing reordering features directly for BLEU with the decoder in the loop. 1 Introduction With the growing adoption of tuning algorithms that can handle thousands of features (Chiang et al., 2008; Hopkins and May, 2011), SMT system de- signers now face a choice when incorporating new ideas into their translation models. Maximum like- lihood models can be estimated from large word- aligned bitexts, creating a small number of highly informative decoder features; or the same ideas can be incorporated into the decoder’s linear model di- rectly. There are trade-offs to each approach. Max- imum likelihood models can be estimated from mil- lions of sentences of bitext, but optimize a mis- matched objective, predicting events observed in word aligned bitext instead of optimizing translation quality. Sparse decoder features have the opposite problem; with the decoder in the loop, we can only tune on small development sets,1 but a translation error metric directly informs training. We investigate this trade-off in the context of re- ordering models for phrase-based decoding. Start- ing with the intuition that most lexicalized reorder- ing models do not smooth their orientation distri- butions intelligently for low-frequency phrase-pairs, we design features that track the first and last words (or clusters) of the phrases in a pair. These features are incorporated into a maximum entropy reorder- ing model, as well as sparse decoder features, to see which approach best complements the now-standard relative-frequency lexicalized reordering model. We also view our work as an example of strong sparse features for phrase-based translation. Fea- tures from hierarchical and syntax-based transla- tion (Chiang et al., 2009) do not easily transfer to the phrase-based paradigm, and most work that has looked at large feature counts in the context of phrase-based translation has focused on the learn- ing method, and not the features themselves (Hop- kins and May, 2011; Cherry and Foster, 2012; Gim- pel and Smith, 2012). We show that by targeting reordering, large gains can be made with relatively simple features. …

Content truncated to 500 words