An Empirical Evaluation of Knowledge Sources and Learning Algorithms for Word Sense Disambiguation

An Empirical Evaluation of Knowledge Sources and Learning Algorithms for Word Sense Disambiguation Yoong Keok Lee and Hwee Tou Ng Department of Computer Science School of Computing National University of Singapore 3 Science Drive 2, Singapore 117543 � leeyoong, nght � @comp.nus.edu.sg Abstract In this paper, we evaluate a vari- ety of knowledge sources and super- vised learning algorithms for word sense disambiguation on SENSEVAL-2 and SENSEVAL-1 data. Our knowledge sources include the part-of-speech of neighboring words, single words in the surrounding context, local collocations, and syntactic relations. The learning al- gorithms evaluated include Support Vec- tor Machines (SVM), Naive Bayes, Ad- aBoost, and decision tree algorithms. We present empirical results showing the rela- tive contribution of the component knowl- edge sources and the different learning algorithms. In particular, using all of these knowledge sources and SVM (i.e., a single learning algorithm) achieves ac- curacy higher than the best official scores on both SENSEVAL-2 and SENSEVAL-1 test data. 1 Introduction Natural language is inherently ambiguous. A word can have multiple meanings (or senses). Given an occurrence of a word � in a natural language text, the task of word sense disambiguation (WSD) is to determine the correct sense of � in that context. WSD is a fundamental problem of natural language processing. For example, effective WSD is crucial for high quality machine translation. One could envisage building a WSD system us- ing handcrafted rules or knowledge obtained from linguists. Such an approach would be highly labor- intensive, with questionable scalability. Another ap- proach involves the use of dictionary or thesaurus to perform WSD. In this paper, we focus on a corpus-based, super- vised learning approach. In this approach, to disam- biguate a word � , we first collect training texts in which instances of � occur. Each occurrence of is manually tagged with the correct sense. We then� train a WSD classifier based on these sample texts, such that the trained classifier is able to assign the sense of � in a new context. Two WSD evaluation exercises, SENSEVAL-1 (Kilgarriff and Palmer, 2000) and SENSEVAL-2 (Edmonds and Cotton, 2001), were conducted in 1998 and 2001, respectively. The lexical sample task in these two SENSEVALs focuses on evalu- ating WSD systems in disambiguating a subset of nouns, verbs, and adjectives, for which manually sense-tagged training data have been collected. In this paper, we conduct a systematic evaluation of the various knowledge sources and supervised learning algorithms on the English lexical sample data sets of both SENSEVALs. 2 Related Work There is a large body of prior research on WSD. Due to space constraints, we will only highlight prior re- search efforts that have investigated (1) contribution of various knowledge sources, or (2) relative perfor- mance of different learning algorithms. Early research efforts on comparing different Association for Computational Linguistics. Language Processing (EMNLP), Philadelphia, July 2002, pp. 41-48. Proceedings of the Conference on Empirical Methods in Natural learning algorithms (Mooney, 1996; Pedersen and Bruce, 1997) tend to base their comparison on only …

Content truncated to 500 words