Optimizing feature set for Chinese Word Sense Disambiguation

Optimizing Feature Set for Chinese Word Sense Disambiguation Zheng-Yu Niu, Dong-Hong Ji Institute for Infocomm Research 21 Heng Mui Keng Terrace 119613 Singapore {zniu, dhji}@i2r.a-star.edu.sg Chew-Lim Tan Department of Computer Science National University of Singapore 3 Science Drive 2 117543 Singapore tancl@comp.nus.edu.sg Abstract This article describes the implementation of I2R word sense disambiguation system (I2R − WSD) that participated in one senseval3 task: Chinese lex- ical sample task. Our core algorithm is a supervised Naive Bayes classifier. This classifier utilizes an op- timal feature set, which is determined by maximiz- ing the cross validated accuracy of NB classifier on training data. The optimal feature set includes part- of-speech with position information in local con- text, and bag of words in topical context. 1 Introduction Word sense disambiguation (WSD) is to assign ap- propriate meaning to a given ambiguous word in a text. Corpus based method is one of the suc- cessful lines of research on WSD. Many supervised learning algorithms have been applied for WSD, ex. Bayesian learning (Leacock et al., 1998), ex- emplar based learning (Ng and Lee, 1996), decision list (Yarowsky, 2000), neural network (Towel and Voorheest, 1998), maximum entropy method (Dang et al., 2002), etc.. In this paper, we employ Naive Bayes classifier to perform WSD. Resolving the ambiguity of words usually relies on the contexts of their occurrences. The feature set used for context representation consists of lo- cal and topical features. Local features include part of speech tags of words within local context, mor- phological information of target word, local collo- cations, and syntactic relations between contextual words and target word, etc.. Topical features are bag of words occurred within topical context. Con- textual features play an important role in providing discrimination information for classifiers in WSD. In other words, an informative feature set will help classifiers to accurately disambiguate word senses, but an uninformative feature set will deteriorate the performance of classifiers. In this paper, we opti- mize feature set by maximizing the cross validated accuracy of Naive Bayes classifier on sense tagged training data. 2 Naive Bayes Classifier Let C = {c1, c2, ..., cL} represent class labels, F = {f1, f2, ..., fM} be a set of features. The value of fj, 1 ≤ j ≤ M, is 1 if fj is present in the context of target word, otherwise 0. In classi- fication process, the Naive Bayes classifier tries to find the class that maximizes P(ci|F), the proba- bility of class ci given feature set F, 1 ≤ i ≤ L. Assuming the independence between features, the classification procedure can be formulated as: ˆi = arg max 1≤i≤L p(ci) �M j=1 p(fj|ci) �M j=1 p(fj) , (1) where p(ci), p(fj|ci) and p(fj) are estimated using maximum likelihood method. To avoid the effects of zero counts when estimating p(fj|ci), the zero counts of p(fj|ci) are replaced with p(ci)/N, where N is the number of training examples. 3 Feature Set For Chinese WSD, there are two strategies to extract contextual information. One is …

Content truncated to 500 words