Improving Statistical Machine Translation Using Word Sense Disambiguation

Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 61–72, Prague, June 2007. c⃝2007 Association for Computational Linguistics Improving Statistical Machine Translation using Word Sense Disambiguation Marine CARPUAT Dekai WU∗ marine@cs.ust.hk dekai@cs.ust.hk Human Language Technology Center HKUST Department of Computer Science and Engineering University of Science and Technology, Clear Water Bay, Hong Kong Abstract We show for the first time that incorporating the predictions of a word sense disambigua- tion system within a typical phrase-based statistical machine translation (SMT) model consistently improves translation quality across all three different IWSLT Chinese- English test sets, as well as producing sta- tistically significant improvements on the larger NIST Chinese-English MT task— and moreover never hurts performance on any test set, according not only to BLEU but to all eight most commonly used au- tomatic evaluation metrics. Recent work has challenged the assumption that word sense disambiguation (WSD) systems are useful for SMT. Yet SMT translation qual- ity still obviously suffers from inaccurate lexical choice. In this paper, we address this problem by investigating a new strat- egy for integrating WSD into an SMT sys- tem, that performs fully phrasal multi-word disambiguation. Instead of directly incor- porating a Senseval-style WSD system, we redefine the WSD task to match the ex- act same phrasal translation disambiguation task faced by phrase-based SMT systems. Our results provide the first known empir- ical evidence that lexical semantics are in- deed useful for SMT, despite claims to the contrary. ∗This material is based upon work supported in part by the Defense Advanced Research Projects Agency (DARPA) under GALE Contract No. HR0011-06-C-0023, and by the Hong Kong Research Grants Council (RGC) research grants 1 Introduction Common assumptions about the role and useful- ness of word sense disambiguation (WSD) models in full-scale statistical machine translation (SMT) systems have recently been challenged. On the one hand, in previous work (Carpuat and Wu, 2005b) we obtained disappointing results when using the predictions of a Senseval WSD system in conjunction with a standard word-based SMT sys- tem: we reported slightly lower BLEU scores de- spite trying to incorporate WSD using a number of apparently sensible methods. These results cast doubt on the assumption that sophisticated dedicated WSD systems that were developed independently from any particular NLP application can easily be integrated into a SMT system so as to improve trans- lation quality through stronger models of context and rich linguistic information. Rather, it has been argued, SMT systems have managed to achieve sig- nificant improvements in translation quality without directly addressing translation disambiguation as a WSD task. Instead, translation disambiguation deci- sions are made indirectly, typically using only word surface forms and very local contextual information, forgoing the much richer linguistic information that WSD systems typically take advantage of. On the other hand, error analysis reveals that the performance of SMT systems still suffers from inac- curate lexical choice. In subsequent empirical stud- ies, we have shown that SMT systems perform much worse than …

Content truncated to 500 words