Meaningful Clustering of Senses Helps Boost Word Sense Disambiguation Performance

Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 105–112, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Meaningful Clustering of Senses Helps Boost Word Sense Disambiguation Performance Roberto Navigli Dipartimento di Informatica Universit`a di Roma “La Sapienza” Roma, Italy navigli@di.uniroma1.it Abstract Fine-grained sense distinctions are one of the major obstacles to successful Word Sense Disambiguation. In this paper, we present a method for reducing the granularity of the WordNet sense inven- tory based on the mapping to a manually crafted dictionary encoding sense hierar- chies, namely the Oxford Dictionary of English. We assess the quality of the map- ping and the induced clustering, and eval- uate the performance of coarse WSD sys- tems in the Senseval-3 English all-words task. 1 Introduction Word Sense Disambiguation (WSD) is undoubt- edly one of the hardest tasks in the field of Nat- ural Language Processing. Even though some re- cent studies report benefits in the use of WSD in specific applications (e.g. Vickrey et al. (2005) and Stokoe (2005)), the present performance of the best ranking WSD systems does not provide a sufficient degree of accuracy to enable real-world, language-aware applications. Most of the disambiguation approaches adopt the WordNet dictionary (Fellbaum, 1998) as a sense inventory, thanks to its free availability, wide coverage, and existence of a number of standard test sets based on it. Unfortunately, WordNet is a fine-grained resource, encoding sense distinctions that are often difficult to recognize even for human annotators (Edmonds and Kilgariff, 1998). Recent estimations of the inter-annotator agree- ment when using the WordNet inventory report figures of 72.5% agreement in the preparation of the English all-words test set at Senseval-3 (Sny- der and Palmer, 2004) and 67.3% on the Open Mind Word Expert annotation exercise (Chklovski and Mihalcea, 2002). These numbers lead us to believe that a credible upper bound for unrestricted fine-grained WSD is around 70%, a figure that state-of-the-art automatic systems find it difficult to outperform. Furthermore, even if a system were able to exceed such an upper bound, it would be unclear how to interpret such a result. It seems therefore that the major obstacle to ef- fective WSD is the fine granularity of the Word- Net sense inventory, rather than the performance of the best disambiguation systems. Interestingly, Ng et al. (1999) show that, when a coarse-grained sense inventory is adopted, the increase in inter- annotator agreement is much higher than the re- duction of the polysemy degree. Following these observations, the main ques- tion that we tackle in this paper is: can we pro- duce and evaluate coarse-grained sense distinc- tions and show that they help boost disambigua- tion on standard test sets? We believe that this is a crucial research topic in the field of WSD, that could potentially benefit several application areas. The contribution of this paper is two-fold. First, we provide a wide-coverage method for clustering WordNet senses via a mapping to a coarse-grained sense inventory, namely the Oxford Dictionary of …

Content truncated to 500 words