Similarity-Based Methods for Word Sense Disambiguation

Similarity-Based Methods For Word Sense Disambiguation Ido Dagan Dept. of Mathematics and Computer Science Bar Ilan University Ramat Gan 52900, Israel dagan©macs, biu. ac. il Lillian Lee Fernando Pereira Div. of Engineering and AT&T Labs - Research Applied Sciences 600 Mountain Ave. Harvard University Murray Hill, NJ 07974, USA Cambridge, MA 01238, USA pereira©research, att. corn llee©eecs, harvard, edu Abstract We compare four similarity-based esti- mation methods against back-off and maximum-likelihood estimation meth- ods on a pseudo-word sense disam- biguation task in which we controlled for both unigram and bigram fre- quency. The similarity-based meth- ods perform up to 40% better on this particular task. We also conclude that events that occur only once in the training set have major impact on similarity-based estimates. 1 Introduction The problem of data sparseness affects all sta- tistical methods for natural language process- ing. Even large training sets tend to misrep- resent low-probability events, since rare events may not appear in the training corpus at all. We concentrate here on the problem of es- timating the probability of unseen word pairs, that is, pairs that do not occur in the train- ing set. Katz's back-off scheme (Katz, 1987), widely used in bigram language modeling, esti- mates the probability of an unseen bigram by utilizing unigram estimates. This has the un- desirable result of assigning unseen bigrams the same probability if they are made up of uni- grams of the same frequency. Class-based methods (Brown et al., 1992; Pereira, Tishby, and Lee, 1993; Resnik, 1992) cluster words into classes of similar words, so that one can base the estimate of a word pair's probability on the averaged cooccurrence prob- ability of the classes to which the two words be- long. However, a word is therefore modeled by the average behavior of many words, which may cause the given word's idiosyncrasies to be ig- nored. For instance, the word "red" might well act like a generic color word in most cases, but it has distinctive cooccurrence patterns with re- spect to words like "apple," "banana," and so on. We therefore consider similarity-based esti- mation schemes that do not require building general word classes. Instead, estimates for the most similar words to a word w are com- bined; the evidence provided by word w' is weighted by a function of its similarity to w. Dagan, Markus, and Markovitch (1993) pro- pose such a scheme for predicting which un- seen cooccurrences are more likely than others. However, their scheme does not assign probabil- ities. In what follows, we focus on probabilistic similarity-based estimation methods. We compared several such methods, in- cluding that of Dagan, Pereira, and Lee (1994) and the cooccurrence smoothing method of Essen and Steinbiss (1992), against classical es- timation methods, including that of Katz, in a decision task involving unseen pairs of direct ob- jects and verbs, where unigram frequency was eliminated from being a factor. We found that all the similarity-based schemes performed al- most 40% better than back-off, which is ex- pected …

Content truncated to 500 words