Supervised Sense Tagging using Support Vector Machines

Supervised Sense Tagging using Support Vector Machines Clara Cabezas, Philip Resnik, and Jessica Stevens Dept. of Linguistics and Institute for Advanced Computer Studies University of Maryland, College Park, MD 20742 USA {clarac,resnik,stevenjc}@umiacs.umd.edu Abstract We describe the University of Maryland's su- pervised sense tagger, which participated in the SENSEVAL-2 lexical sample evaluations for En- glish, Spanish, and Swedish; we also present un- official results for Basque. We designed a highly modular combination of language-independent feature extraction and supervised learning us- ing support vector machines in order to permit rapid ramp-up, language independence, and ca- pability for future expansion. 1 Introduction The SENSEVAL-2 exercise provided an unprece- dented opportunity to explore word sense dis- ambiguation (WSD) in a common evaluation framework for a large number of languages. In past work, we have focused on unsupervised methods for English, taking advantage of the WordN et hierarchy and sometimes also selec- tional preferences between predicates and argu- ments (Resnik, 1997; Resnik, 1999). In the cur- rent exercise, however, WordNet-like sense hi- erarchies were not necessarily going to be avail- able for all languages, and the predominance of lexical selection tasks (rather than all-words tasks) suggested adopting a disambiguation ap- proach capable of exploiting manually anno- tated training data. These considerations mo- tivated a system design based on supervised learning, where senses to be predicted did not need to be treated as part of a semantic hierar- chy. Our design was also motivated by the role of semantic selection techniques in our longer term research agenda. In the context of our group's work on cross-language information retrieval and machine translation applications (Resnik et -al., 2001; Cabezas et al., 2001), lexical selection that is, choosing the right target-language 59 -word given a source-language word in context is a crucial task. Because the lexical selec- tion problem is extremely similar to sense selec- tion, and because this was our first foray into supervised methods, we took advantage of the opportunity to construct an architecture that will support both tasks. In the sections that follow, we lay out our system architecture, briefly summarize our SENSEVAL-2 results, and discuss our plans for future work. 2 System Architecture UMD's system follows the classic supervised learning paradigm that, for WSD, is perhaps best exemplified by Yarowsky's (1993) work. Each word in the vocabulary is considered an independent classification problem. First, an- notated training instances for the ambiguous word are analyzed so that each instance can be represented as a collection of feature-value pairs labeled with the correct category. Then, these data are used for parameter estimation within a supervised learning framework in or- der to produce a trained classifier. Finally, the trained classifier is given previously unseen test instances and for each instance it predicts what the appropriate category label should be. 2.1 Contextual Features We began by tokenizing all the training in- stances using a simple language-specific tok- enizer. Features were then defined in terms of the presence of tokens either within a wide con- text or at a certain …

Content truncated to 500 words