Learning Word Sense With Feature Selection and Order Identification Capabilities

Learning Word Senses With Feature Selection and Order Identification Capabilities Zheng-Yu Niu, Dong-Hong Ji Institute for Infocomm Research 21 Heng Mui Keng Terrace 119613 Singapore {zniu, dhji}@i2r.a-star.edu.sg Chew-Lim Tan Department of Computer Science National University of Singapore 3 Science Drive 2 117543 Singapore tancl@comp.nus.edu.sg Abstract This paper presents an unsupervised word sense learning algorithm, which induces senses of target word by grouping its occurrences into a “natural” number of clusters based on the similarity of their contexts. For removing noisy words in feature set, feature selection is conducted by optimizing a clus- ter validation criterion subject to some constraint in an unsupervised manner. Gaussian mixture model and Minimum Description Length criterion are used to estimate cluster structure and cluster number. Experimental results show that our algorithm can find important feature subset, estimate model or- der (cluster number) and achieve better performance than another algorithm which requires cluster num- ber to be provided. 1 Introduction Sense disambiguation is essential for many lan- guage applications such as machine translation, in- formation retrieval, and speech processing (Ide and V´eronis, 1998). Almost all of sense disambigua- tion methods are heavily dependant on manually compiled lexical resources. However these lexical resources often miss domain specific word senses, even many new words are not included inside. Learning word senses from free text will help us dispense of outside knowledge source for defining sense by only discriminating senses of words. An- other application of word sense learning is to help enriching or even constructing semantic lexicons (Widdows, 2003). The solution of word sense learning is closely re- lated to the interpretation of word senses. Different interpretations of word senses result in different so- lutions to word sense learning. One interpretation strategy is to treat a word sense as a set of synonyms like synset in WordNet. The committee based word sense discovery algorithm (Pantel and Lin, 2002) followed this strategy, which treated senses as clusters of words occurring in sim- ilar contexts. Their algorithm initially discovered tight clusters called committees by grouping top n words similar with target word using average- link clustering. Then the target word was assigned to committees if the similarity between them was above a given threshold. Each committee that the target word belonged to was interpreted as one of its senses. There are two difficulties with this committee based sense learning. The first difficulty is about derivation of feature vectors. A feature for target word here consists of a contextual content word and its grammatical relationship with target word. Ac- quisition of grammatical relationship depends on the output of a syntactic parser. But for some lan- guages, ex. Chinese, the performance of syntactic parsing is still a problem. The second difficulty with this solution is that two parameters are required to be provided, which control the number of commit- tees and the number of senses of target word. Another interpretation strategy is to treat a word sense as a group of similar contexts of target word. The context group discrimination (CGD) algorithm …

Content truncated to 500 words