Word Independent Context Pair Classification Model for Word Sense Disambiguation

Proceedings of the 9th Conference on Computational Natural Language Learning (CoNLL), pages 33–39, Ann Arbor, June 2005. c⃝2005 Association for Computational Linguistics Abstract Traditionally, word sense disambiguation (WSD) involves a different context classifi- cation model for each individual word. This paper presents a weakly supervised learning approach to WSD based on learning a word independent context pair classification model. Statistical models are not trained for classifying the word contexts, but for classi- fying a pair of contexts, i.e. determining if a pair of contexts of the same ambiguous word refers to the same or different senses. Using this approach, annotated corpus of a target word A can be explored to disambiguate senses of a different word B. Hence, only a limited amount of existing annotated corpus is required in order to disambiguate the entire vocabulary. In this research, maximum en- tropy modeling is used to train the word in- dependent context pair classification model. Then based on the context pair classification results, clustering is performed on word men- tions extracted from a large raw corpus. The resulting context clusters are mapped onto the external thesaurus WordNet. This ap- proach shows great flexibility to efficiently integrate heterogeneous knowledge sources, e.g. trigger words and parsing structures. Based on Senseval-3 Lexical Sample stan- dards, this approach achieves state-of-the-art performance in the unsupervised learning category, and performs comparably with the supervised Naïve Bayes system. 1 Introduction Word Sense Disambiguation (WSD) is one of the central problems in Natural Language Processing. The difficulty of this task lies in the fact that con- text features and the corresponding statistical dis- tribution are different for each individual word. Traditionally, WSD involves training the context classification models for each ambiguous word. (Gale et al. 1992) uses the Naïve Bayes method for context classification which requires a manually annotated corpus for each ambiguous word. This causes a serious Knowledge Bottleneck. The bot- tleneck is particularly serious when considering the domain dependency of word senses. To overcome the Knowledge Bottleneck, unsupervised or weakly supervised learning approaches have been pro- posed. These include the bootstrapping approach (Yarowsky 1995) and the context clustering ap- proach (Schütze 1998). The above unsupervised or weakly supervised learning approaches are less subject to the Knowl- edge Bottleneck. For example, (Yarowsky 1995) only requires sense number and a few seeds for each sense of an ambiguous word (hereafter called keyword). (Schütze 1998) may only need minimal annotation to map the resulting context clusters onto external thesaurus for benchmarking and ap- plication-related purposes. Both methods are based on trigger words only. This paper presents a novel approach based on learning word-independent context pair classifica- tion model. This idea may be traced back to (Schütze 1998) where context clusters based on generic Euclidean distance are regarded as distinct word senses. Different from (Schütze 1998), we observe that generic context clusters may not al- ways correspond to distinct word senses. There- fore, we used supervised machine learning to model the relationships between the context dis- tinctness and the sense distinctness. …

Content truncated to 500 words