Effective Use of Linguistic and Contextual Information for Statistical Machine Translation

Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 72–80, Singapore, 6-7 August 2009. c⃝2009 ACL and AFNLP Effective Use of Linguistic and Contextual Information for Statistical Machine Translation Libin Shen and Jinxi Xu and Bing Zhang and Spyros Matsoukas and Ralph Weischedel BBN Technologies Cambridge, MA 02138, USA {lshen,jxu,bzhang,smatsouk,weisched}@bbn.com Abstract Current methods of using lexical features in machine translation have difficulty in scaling up to realistic MT tasks due to a prohibitively large number of parame- ters involved. In this paper, we propose methods of using new linguistic and con- textual features that do not suffer from this problem and apply them in a state-of- the-art hierarchical MT system. The fea- tures used in this work are non-terminal labels, non-terminal length distribution, source string context and source depen- dency LM scores. The effectiveness of our techniques is demonstrated by signif- icant improvements over a strong base- line. On Arabic-to-English translation, improvements in lower-cased BLEU are 2.0 on NIST MT06 and 1.7 on MT08 newswire data on decoding output. On Chinese-to-English translation, the im- provements are 1.0 on MT06 and 0.8 on MT08 newswire data. 1 Introduction Linguistic and context features, especially sparse lexical features, have been widely used in re- cent machine translation (MT) research. Unfor- tunately, existing methods of using such features are not ideal for large-scale, practical translation tasks. In this paper, we will propose several prob- abilistic models to effectively exploit linguistic and contextual information for MT decoding, and these new features do not suffer from the scalabil- ity problem. Our new models are tested on NIST MT06 and MT08 data, and they provide signifi- cant improvement over a strong baseline system. 1.1 Previous Work The ideas of using labels, length preference and source side context in MT decoding were explored previously. Broadly speaking, two approaches were commonly used in existing work. One is to use a stochastic gradient descent (SGD) or Perceptron like online learning algo- rithm to optimize the weights of these features directly for MT (Shen et al., 2004; Liang et al., 2006; Tillmann and Zhang, 2006). This method is very attractive, since it opens the door to rich lex- ical features. However, in order to robustly opti- mize the feature weights, one has to use a substan- tially large development set, which results in sig- nificantly slower tuning. Alternatively, one needs to carefully select a development set that simulates the test set to reduce the risk of over-fitting, which however is not always realistic for practical use. A remedy is to aggressively limit the feature space, e.g. to syntactic labels or a small fraction of the bi-lingual features available, as in (Chiang et al., 2008; Chiang et al., 2009), but that reduces the benefit of lexical features. A possible generic solution is to cluster the lexical features in some way. However, how to make it work on such a large space of bi-lingual features is still an open question. The other approach is to estimate a single score …

Content truncated to 500 words