Word Sense Disambiguation vs. Statistical Machine Translation

Proceedings of the 43rd Annual Meeting of the ACL, pages 387–394, Ann Arbor, June 2005. c⃝2005 Association for Computational Linguistics Word Sense Disambiguation vs. Statistical Machine Translation Marine CARPUAT Dekai WU1 marine@cs.ust.hk dekai@cs.ust.hk Human Language Technology Center HKUST Department of Computer Science University of Science and Technology Clear Water Bay, Hong Kong Abstract We directly investigate a subject of much recent debate: do word sense disambiga- tion models help statistical machine trans- lation quality? We present empirical re- sults casting doubt on this common, but unproved, assumption. Using a state-of- the-art Chinese word sense disambigua- tion model to choose translation candi- dates for a typical IBM statistical MT system, we find that word sense disam- biguation does not yield significantly bet- ter translation quality than the statistical machine translation system alone. Error analysis suggests several key factors be- hind this surprising finding, including in- herent limitations of current statistical MT architectures. 1 Introduction Word sense disambiguation or WSD, the task of de- termining the correct sense of a word in context, is a much studied problem area with a long and hon- orable history. Recent years have seen steady ac- curacy gains in WSD models, driven in particular by controlled evaluations such as the Senseval series of workshops. Word sense disambiguation is often assumed to be an intermediate task, which should then help higher level applications such as machine 1The authors would like to thank the Hong Kong Re- search Grants Council (RGC) for supporting this research in part through grants RGC6083/99E, RGC6256/00E, and DAG03/04.EG09, and several anonymous reviewers for in- sights and suggestions. translation or information retrieval. However, WSD is usually performed and evaluated as a standalone task, and to date there have been very few efforts to integrate the learned WSD models into full statisti- cal MT systems. An energetically debated question at conferences over the past year is whether even the new state- of-the-art word sense disambiguation models actu- ally have anything to offer to full statistical machine translation systems. Among WSD circles, this can sometimes elicit responses that border on implying that even asking the question is heretical. In efforts such as Senseval we tend to regard the construction of WSD models as an obviously correct, if necessar- ily simplified, approach that will eventually lead to essential disambiguation components within larger applications like machine translation. There is no question that the word sense disam- biguation perspective has led to numerous insights in machine translation, even of the statistical variety. It is often simply an unstated assumption that any full translation system, to achieve full performance, will sooner or later have to incorporate individual WSD components. However, in some translation architectures and particularly in statistical machine translation (SMT), the translation engine already implicitly factors in many contextual features into lexical choice. From this standpoint, SMT models can be seen as WSD models in their own right, albeit with several major caveats. But typical statistical machine translation models only rely on a local context to choose among lexical …

Content truncated to 500 words