Senseval-3 task: Word Sense Disambiguation of WordNet glosses

SENSEVAL-3 TASK Word-Sense Disambiguation of WordNet Glosses Kenneth C. Litkowski CL Research 9208 Gue Road Damascus, MD 20872 ken@clres.com Abstract The SENSEVAL-3 task to perform word-sense disambiguation of WordNet glosses was designed to encourage development of technology to make use of standard lexical resources. The task was based on the availability of sense- disambiguated hand-tagged glosses created in the eXtended WordNet project. The hand-tagged glosses provided a “gold standard” for judging the performance of automated disambiguation systems. Seven teams participated in the task, with a total of 10 runs. Scoring these runs as an “all-words” task, along with considerable discussions among participants, provided more insights than just the underlying technology. The task identified several issues about the nature of the WordNet sense inventory and the underlying use of wordnet design principles, particularly the significance of WordNet-style relations. Introduction In SENSEVAL-2, performance in the lexical sample task dropped considerably (Kilgarriff, 2001). Kilgarriff suggested that using WordNet (Fellbaum, 1998) for SENSEVAL has drawbacks. WordNet was not designed to serve as a lexical resource, but its public availability and reasonable comprehensiveness have been dominant factors in its selection as the lexical resource of choice for Senseval and for many applications. These factors have led to further funding by U.S. government agencies and many improvements are currently underway. Among these improvements is a planned hand-tagging of the WordNet glosses with their WordNet senses. At the same time, sense-tagging of the glosses is being performed in the Extended WordNet (XWN) project under development at the University of Texas at Dallas (Mihalcea and Moldovan, 2001)1. The XWN project also parses the WordNet glosses into a part of speech tree and transforms them into a logical predicate form. More generally, sense disambiguation of definitions in any lexical resource is an important objective in the language engineering community. The first significant disambiguation of dictionary definitions and creation of a hierarchy took place 25 years ago in the groundbreaking work of Amsler (1980). However, while substantial research has been performed on machine-readable dictionaries since that time, technology has not yet been developed to make systematic use of these resources. This SENSEVAL task was designed to encourage the lexical research community to take up the challenge of disambiguating dictionary definitions. XWN is used as a core knowledge base for applications such as question answering, information retrieval, information extraction, summarization, natural language generation, inferences, and other knowledge intensive applications. The glosses contain a part of the world knowledge since they define the most common concepts of the English language. In the XWN project, many open-class words in WordNet glosses have been hand-tagged and provide a “gold standard” against which disambiguation systems can be judged. The SENSEVAL-3 task is to replicate the hand-tagged results. The Extended WordNet (XWN) project has disambiguated the content words (nouns, verbs, adjectives, and adverbs) of all glosses, combining human annotation and automated methods using WordNet 1.7.1. A “quality” attribute was given to each lemma. XWN used two automatic systems to disambiguate the content words. When the two …

Content truncated to 500 words