Word Sense Disambiguation Using OntoNotes: An Empirical Study

Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 1002–1010, Honolulu, October 2008. c⃝2008 Association for Computational Linguistics Word Sense Disambiguation Using OntoNotes: An Empirical Study Zhi Zhong and Hwee Tou Ng and Yee Seng Chan Department of Computer Science National University of Singapore Law Link, Singapore 117590 {zhongzhi, nght, chanys}@comp.nus.edu.sg Abstract The accuracy of current word sense disam- biguation (WSD) systems is affected by the fine-grained sense inventory of WordNet as well as a lack of training examples. Using the WSD examples provided through OntoNotes, we conduct the first large-scale WSD evalua- tion involving hundreds of word types and tens of thousands of sense-tagged examples, while adopting a coarse-grained sense inventory. We show that though WSD systems trained with a large number of examples can obtain a high level of accuracy, they nevertheless suffer a substantial drop in accuracy when applied to a different domain. To address this issue, we propose combining a domain adaptation tech- nique using feature augmentation with active learning. Our results show that this approach is effective in reducing the annotation effort required to adapt a WSD system to a new do- main. Finally, we propose that one can maxi- mize the dual benefits of reducing the annota- tion effort while ensuring an increase in WSD accuracy, by only performing active learning on the set of most frequently occurring word types. 1 Introduction In language, many words have multiple meanings. The process of identifying the correct meaning, or sense of a word in context, is known as word sense disambiguation (WSD). WSD is one of the funda- mental problems in natural language processing and is important for applications such as machine trans- lation (MT) (Chan et al., 2007a; Carpuat and Wu, 2007), information retrieval (IR), etc. WSD is typically viewed as a classification prob- lem where each ambiguous word is assigned a sense label (from a pre-defined sense inventory) during the disambiguation process. In current WSD research, WordNet (Miller, 1990) is usually used as the sense inventory. WordNet, however, adopts a very fine level of sense granularity, thus restricting the accu- racy of WSD systems. Also, current state-of-the-art WSD systems are based on supervised learning and face a general lack of training data. To provide a standardized test-bed for evalua- tion of WSD systems, a series of evaluation exer- cises called SENSEVAL were held. In the English all-words task of SENSEVAL-2 and SENSEVAL- 3 (Palmer et al., 2001; Snyder and Palmer, 2004), no training data was provided and systems must tag all the content words (noun, verb, adjective, and adverb) in running English texts with their correct WordNet senses. In SENSEVAL-2, the best per- forming system (Mihalcea and Moldovan, 2001) in the English all-words task achieved an accuracy of 69.0%, while in SENSEVAL-3, the best perform- ing system (Decadt et al., 2004) achieved an accu- racy of 65.2%. In SemEval-2007, which was the most recent SENSEVAL evaluation, a similar En- glish all-words task was held, where systems had to provide the …

Content truncated to 500 words