Sequential Model Selection for Word Sense Disambiguation

Sequential Model Selection for Word Sense Disambiguation * Ted Pedersen t and Rebecca Brucet and Janyce Wiebe:~ tDepartment of Computer Science and Engineering Southern Methodist University, Dallas, TX 75275 :IDepartment of Computer Science New Mexico State University, Las Cruces, NM 88003 pedersen@seas, smu. edu, rbruce@seas, smu. edu, wiebe~cs, nmsu. edu Abstract Statistical models of word-sense disam- biguation are often based on a small num- ber of contextual features or on a model that is assumed to characterize the inter- actions among a set of features. Model selection is presented as an alternative to these approaches, where a sequential search of possible models is conducted in order to find the model that best characterizes the interactions among features. This paper expands existing model selection method- ology and presents the first comparative study of model selection search strategies and evaluation criteria when applied to the problem of building probabilistic classifiers for word-sense disambiguation. 1 Introduction In this paper word-sense disambiguation is cast as a problem in supervised learning, where a classifier is induced from a corpus of sense-tagged text. Sup- pose there is a training sample where each sense- tagged sentence is represented by the feature vari- ables (F1,..., Fn-1, S). Selected contextual proper- ties of the sentence are represented by (F1, • •., Fn-1) and the sense of the ambiguous word is represented by S. Our task is to induce a classifier that will predict the value of S given an untagged sentence represented by the contextual feature variables. We adopt a statistical approach whereby a prob- abilistic model is selected that describes the inter- actions among the feature variables. Such a model can form the basis of a probabilistic classifier since it specifies the probability of observing any and all combinations of the values of the feature variables. Suppose our training sample has N sense-tagged sentences. There are q possible combinations of val- ues for the n feature variables, where each such com- bination is represented by a feature vector. Let • This research was supported by the Office of Naval Research under grant number N00014-95-1-0776. fi and Oi be the frequency and probability of ob- serving the i th feature vector, respectively. Then (fl,..., fq) has a multinomial distribution with pa- rameters (N, 81,..., 8q). The 0 parameters, i.e., the joint parameters, define the joint probability distri- bution of the feature variables. These are the pa- rameters of the fully saturated model, the model in which the value of each variable directly affects the values of all the other variables. These parameters can be estimated as maximum likelihood estimates (MLEs), such that the estimate of 8i, ~/, is ~. For these estimates to be reliable, each of the q possible combinations of feature values must occur in the training sample. This is unlikely for NLP data samples, which are often sparse and highly skewed (c.f., e.g. (Pedersen et al., 1996) and (Zipf, 1935)). However, if the data sample can be adequately characterized by a less complex model, i.e., a …

Content truncated to 500 words