PNNL: A Supervised Maximum Entropy Approach to Word Sense Disambiguation

Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007), pages 264–267, Prague, June 2007. c⃝2007 Association for Computational Linguistics PNNL: A Supervised Maximum Entropy Approach to Word Sense Disambiguation Stephen Tratz, Antonio Sanfilippo, Michelle Gregory, Alan Chappell, Christian Posse, Paul Whitney Pacific Northwest National Laboratory 902 Battelle Blvd, PO Box 999 Richland, WA 99352, USA {stephen.tratz, antonio.sanfilippo, michelle, alan.chap- pell, christian.posse, paul.whitney}@pnl.gov Abstract In this paper, we described the PNNL Word Sense Disambiguation system as ap- plied to the English all-word task in Se- mEval 2007. We use a supervised learning approach, employing a large number of features and using Information Gain for di- mension reduction. The rich feature set combined with a Maximum Entropy classi- fier produces results that are significantly better than baseline and are the highest F- score for the fined-grained English all- words subtask of SemEval. 1 Introduction Accurate word sense disambiguation (WSD) can support many natural language processing and knowledge management tasks. The main goal of the PNNL WSD system is to support Semantic Web applications, such as semantic-driven search and navigation, through a reliable mapping of words in naturally occurring text to ontological classes. As described in Sanfilippo et al. (2006), this goal is achieved by defining a WordNet-based (Fellbaum, 1998) ontology that offers a manage- able set of concept classes, provides an extensive characterization of concept class in terms of lexical instances, and integrates an automated class recog- nition algorithm. We found that the same features that are useful for predicting word classes are also useful in distinguishing individual word senses. Our main objective in this paper is to predict in- dividual word senses using a large combination of features including contextual, semantic, and syn- tactic information. In our earlier paper (Sanfilippo et al., 2006), we reported that the PNNL WSD sys- tem exceeded the performance of the best perform- ers for verbs in the SENSEVAL-3 English all- words task dataset. SemEval 2007 is our first op- portunity to enter a word sense disambiguation competition. 2 Approach While many unsupervised word sense disambigua- tion systems have been created, supervised systems have generally produced superior results (Snyder and Palmer, 2004; Mihalcea et al., 2004). Our sys- tem is based on a supervised WSD approach that uses a Maximum Entropy classifier to predict WordNet senses. We use SemCor1, OMWE 1.0 (Chklovski and Mihalcea, 2002), and example sentences in Word- Net as the training corpus. We utilize the OpenNLP MaxEnt implementation2 of the maxi- mum entropy classification algorithm (Berger et al., 1996) to train classification models for each lemma and part-of-speech combination in the train- ing corpus. These models are used to predict WordNet senses for words found in natural text. For lemma and part-of-speech combinations that are not present in the training corpus, the PNNL WSD system defaults to the most frequent Word- Net sense. 2.1 Features We use a rich set of features to predict individual word senses. A large number of features are ex- tracted for each word sense instance in the …

Content truncated to 500 words