Improving Word Sense Disambiguation Using Topic Features

Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 1015–1023, Prague, June 2007. c⃝2007 Association for Computational Linguistics Improving Word Sense Disambiguation Using Topic Features Jun Fu Cai, Wee Sun Lee Department of Computer Science National University of Singapore 3 Science Drive 2, Singapore 117543 {caijunfu, leews}@comp.nus.edu.sg Yee Whye Teh Gatsby Computational Neuroscience Unit University College London 17 Queen Square, London WC1N 3AR, UK ywteh@gatsby.ucl.ac.uk Abstract This paper presents a novel approach for ex- ploiting the global context for the task of word sense disambiguation (WSD). This is done by using topic features constructed us- ing the latent dirichlet allocation (LDA) al- gorithm on unlabeled data. The features are incorporated into a modified na¨ıve Bayes network alongside other features such as part-of-speech of neighboring words, single words in the surrounding context, local col- locations, and syntactic patterns. In both the English all-words task and the English lex- ical sample task, the method achieved sig- nificant improvement over the simple na¨ıve Bayes classifier and higher accuracy than the best official scores on Senseval-3 for both task. 1 Introduction Natural language tends to be ambiguous. A word often has more than one meanings depending on the context. Word sense disambiguation (WSD) is a nat- ural language processing (NLP) task in which the correct meaning (sense) of a word in a given context is to be determined. Supervised corpus-based approach has been the most successful in WSD to date. In such an ap- proach, a corpus in which ambiguous words have been annotated with correct senses is first collected. Knowledge sources, or features, from the context of the annotated word are extracted to form the training data. A learning algorithm, like the support vector machine (SVM) or na¨ıve Bayes, is then applied on the training data to learn the model. Finally, in test- ing, the learnt model is applied on the test data to assign the correct sense to any ambiguous word. The features used in these systems usually in- clude local features, such as part-of-speech (POS) of neighboring words, local collocations , syntac- tic patterns and global features such as single words in the surrounding context (bag-of-words) (Lee and Ng, 2002). However, due to the data scarcity prob- lem, these features are usually very sparse in the training data. There are, on average, 11 and 28 training cases per sense in Senseval 2 and 3 lexi- cal sample task respectively, and 6.5 training cases per sense in the SemCor corpus. This problem is especially prominent for the bag-of-words feature; more than hundreds of bag-of-words are usually ex- tracted for each training instance and each feature could be drawn from any English word. A direct consequence is that the global context information, which the bag-of-words feature is supposed to cap- ture, may be poorly represented. Our approach tries to address this problem by clustering features to relieve the scarcity problem, specifically on the bag-of-words feature. In the pro- cess, we construct topic features, …

Content truncated to 500 words