NUS-ML:Improving Word Sense Disambiguation Using Topic Features

Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007), pages 249–252, Prague, June 2007. c⃝2007 Association for Computational Linguistics NUS-ML: Improving Word Sense Disambiguation Using Topic Features Jun Fu Cai, Wee Sun Lee Department of Computer Science National University of Singapore 3 Science Drive 2, Singapore 117543 {caijunfu, leews}@comp.nus.edu.sg Yee Whye Teh Gatsby Computational Neuroscience Unit University College London 17 Queen Square, London WC1N 3AR, UK ywteh@gatsby.ucl.ac.uk Abstract We participated in SemEval-1 English coarse-grained all-words task (task 7), En- glish fine-grained all-words task (task 17, subtask 3) and English coarse-grained lex- ical sample task (task 17, subtask 1). The same method with different labeled data is used for the tasks; SemCor is the labeled corpus used to train our system for the all- words tasks while the labeled corpus that is provided is used for the lexical sam- ple task. The knowledge sources include part-of-speech of neighboring words, single words in the surrounding context, local col- locations, and syntactic patterns. In addi- tion, we constructed a topic feature, targeted to capture the global context information, using the latent dirichlet allocation (LDA) algorithm with unlabeled corpus. A modi- fied na¨ıve Bayes classifier is constructed to incorporate all the features. We achieved 81.6%, 57.6%, 88.7% for coarse-grained all- words task, fine-grained all-words task and coarse-grained lexical sample task respec- tively. 1 Introduction Supervised corpus-based approach has been the most successful in WSD to date. However, this ap- proach faces severe data scarcity problem, resulting features being sparsely represented in the training data. This problem is especially prominent for the bag-of-words feature. A direct consequence is that the global context information, which the bag-of- words feature is supposed to capture, may be poorly represented. Our system tries to address this problem by clustering features to relieve the scarcity problem, specifically on the bag-of-words feature. In the pro- cess, we construct topic features, trained using the latent dirichlet allocation (LDA) algorithm. We train the topic model (Blei et al., 2003) on unlabeled data, clustering the words occurring in the corpus to a pre- defined number of topics. We then use the resulting topic model to tag the bag-of-words in the labeled corpus with topic distributions. We incorporate the distributions, called the topic features, using a simple Bayesian network, modified from na¨ıve Bayes model, alongside other features and train the model on the labeled corpus. 2 Feature Construction 2.1 Baseline Features For both the lexical sample and all-words tasks, we use the following standard baseline features. POS Tags For each word instance w, we include POS tags for P words prior to as well as after w within the same sentence boundary. We also include the POS tag of w. If there are fewer than P words prior or after w in the same sentence, we denote the corresponding feature as NIL. Local Collocations We adopt the same 11 col- location features as (Lee and Ng, 2002), namely C−1,−1, C1,1, C−2,−2, C2,2, C−2,−1, C−1,1, C1,2, C−3,−1, C−2,1, C−1,2, and C1,3. 249 Bag-of-Words For each …

Content truncated to 500 words