Decision Trees for Lexical Smoothing in Statistical Machine Translation

Proceedings of the Joint 5th Workshop on Statistical Machine Translation and MetricsMATR, pages 428–437, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics Decision Trees for Lexical Smoothing in Statistical Machine Translation Rabih Zbib† and Spyros Matsoukas and Richard Schwartz and John Makhoul BBN Technologies, 10 Moulton Street, Cambridge, MA 02138, USA † Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139, USA Abstract We present a method for incorporat- ing arbitrary context-informed word at- tributes into statistical machine trans- lation by clustering attribute-quali ed source words, and smoothing their word translation probabilities using bi- nary decision trees. We describe two ways in which the decision trees are used in machine translation: by us- ing the attribute-quali ed source word clusters directly, or by using attribute- dependent lexical translation probabil- ities that are obtained from the trees, as a lexical smoothing feature in the de- coder model. We present experiments using Arabic-to-English newswire data, and using Arabic diacritics and part-of- speech as source word attributes, and show that the proposed method im- proves on a state-of-the-art translation system. 1 Introduction Modern statistical machine translation (SMT) models, such as phrase-based SMT or hierar- chical SMT, implicitly incorporate source lan- guage context. It has been shown, however, that such systems can still bene t from the explicit addition of lexical, syntactic or other kinds of context-informed word features (Vick- rey et al., 2005; Gimpel and Smith, 2008; Brunning et al., 2009; Devlin, 2009). But the bene t obtained from the addition of attribute information is in general countered by the in- crease in the model complexity, which in turn results in a sparser translation model when es- timated from the same corpus of data. The increase in model sparsity usually results in a deterioration of translation quality. In this paper, we present a method for using arbitrary types of source-side context-informed word attributes, using binary decision trees to deal with the sparsity side-eect. The deci- sion trees cluster attribute-dependent source words by reducing the entropy of the lexi- cal translation probabilities. We also present another method where, instead of clustering the attribute-dependent source words, the de- cision trees are used to interpolate attribute- dependent lexical translation probability mod- els, and use those probabilities to compute a feature in the decoder log-linear model. The experiments we present in this paper were conducted on the translation of Arabic- to-English newswire data using a hierarchical system based on (Shen et al., 2008), and using Arabic diacritics (see section 2.3) and part-of- speech (POS) as source word attributes. Pre- vious work that attempts to use Arabic dia- critics in machine translation runs against the sparsity problem, and appears to lose most of the useful information contained in the dia- critics when using partial diacritization (Diab et al., 2007). Using the methods proposed in this paper, we manage to obtain consistent improvements from diacritics against a strong baseline. The methods we propose, though, are not restrictive to Arabic-to-English trans- lation. The same techniques can also …

Content truncated to 500 words