Combining Lexical and Syntactic Features for Supervised Word Sense Disambiguation

Combining Lexical and Syntactic Features for Supervised Word Sense Disambiguation Saif Mohammad University of Toronto Toronto, ON M4M2X6 Canada smm@cs.toronto.edu http://www.cs.toronto.edu/˜smm Ted Pedersen University of Minnesota Duluth, MN 55812 USA tpederse@d.umn.edu http://www.d.umn.edu/˜tpederse Abstract The success of supervised learning approaches to word sense disambiguation is largely de- pendent on the features used to represent the context in which an ambiguous word occurs. Previous work has reached mixed conclusions; some suggest that combinations of syntactic and lexical features will perform most effec- tively. However, others have shown that sim- ple lexical features perform well on their own. This paper evaluates the effect of using differ- ent lexical and syntactic features both individu- ally and in combination. We show that it is pos- sible for a very simple ensemble that utilizes a single lexical feature and a sequence of part of speech features to result in disambiguation ac- curacy that is near state of the art. 1 Introduction Most words in natural language exhibit polysemy, that is, they have multiple possible meanings. Each of these meanings is referred to as a sense, and word sense disam- biguation is the process of identifying the intended sense of a target word based on the context in which it is used. The context of the target word consists of the sentence in which it occurs, and possibly one or two surrounding sentences. Consider the following sentence: Harry cast a bewitching spell (1) The target word spell has many possible senses, such as, a charm or incantation, to read out letter by letter, and a period of time. The intended sense, a charm or incan- tation, can be identified based on the context, which in this case includes bewitching and a reference to a famous young wizard. Word sense disambiguation is often approached by su- pervised learning techniques. The training data consists of sentences which have potential target words tagged by a human expert with their intended sense. Numerous learning algorithms, such as, Naive Bayesian classifiers, Decision Trees and Neural Networks have been used to learn models of disambiguation. However, both (Peder- sen, 2001a) and (Lee and Ng, 2002) suggest that different learning algorithms result in little change in overall dis- ambiguation results, and that the real determiner of accu- racy is the set of features that are employed. Previous work has shown that using different combi- nations of features is advantageous for word sense dis- ambiguation (e.g., (McRoy, 1992), (Ng and Lee, 1996), (Stevenson and Wilks, 2001), (Yarowsky and Florian, 2002)). However, less attention is paid to determining what the minimal set of features necessary to attain high accuracy disambiguation are. In this paper we present experiments that measure the redundancy in disambigua- tion accuracy achieved by classifiers using two different sets of features, and we also determine an upper bound on the accuracy that could be attained via the combination of such classifiers into an ensemble. We find that simple combinations of lexical and syn- tactic features can result in very high disambiguation accuracy, via …

Content truncated to 500 words