Graded Word Sense Assignment

Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 440–449, Singapore, 6-7 August 2009. c⃝2009 ACL and AFNLP Graded Word Sense Assignment Katrin Erk University of Texas at Austin katrin.erk@mail.utexas.edu Diana McCarthy University of Sussex dianam@sussex.ac.uk Abstract Word sense disambiguation is typically phrased as the task of labeling a word in context with the best-fitting sense from a sense inventory such as WordNet. While questions have often been raised over the choice of sense inventory, computational linguists have readily accepted the best- fitting sense methodology despite the fact that the case for discrete sense bound- aries is widely disputed by lexical seman- tics researchers. This paper studies graded word sense assignment, based on a recent dataset of graded word sense annotation. 1 Introduction The task of automatically characterizing word meaning in text is typically modeled as word sense disambiguation (WSD): given a list of senses for target lemma w, the task is to pick the best-fitting sense for a given occurrence of w. The list of senses is usually taken from an online dictionary or thesaurus. However, clear cut sense boundaries are sometimes hard to define, and the meaning of words depends strongly on the context in which they are used (Cruse, 2000; Hanks, 2000). Some researchers in lexical semantics have suggested that word meanings lie on a continuum between i) clear cut cases of ambiguity and ii) vagueness where clear cut boundaries do not hold (Tuggy, 1993). Certainly, it seems that a more complex representation of word sense is needed with a softer, graded representation of meaning rather than a fixed listing of senses (Cruse, 2000). A recent annotation study ((Erk et al., 2009), hereafter GWS) marked a target word in context with graded ratings (on a scale of 1-5) on senses from WordNet (Fellbaum, 1998). Table 1 shows an example of a sentence with the target word in bold, and with the annotator judgments given to each sense. The study found that annotators made ample use of the intermediate ratings on the scale, and often gave high ratings to more than one WordNet sense for the same occurrence. It was found that the annotator ratings could not easily be transformed to categorial judgments by making more coarse-grained senses. If human word sense judgments are best viewed as graded, it makes sense to explore models of word sense that can predict graded sense assignments. In this paper we look at the issue of graded ap- plicability of word sense from the point of view of automatic graded word sense assignment, us- ing the GWS graded word sense dataset. We make three primary contributions. Firstly, we propose evaluation metrics that can be used on graded word sense judgments. Some of these metrics, like Spearman’s ρ, have been used previously (Mc- Carthy et al., 2003; Mitchell and Lapata, 2008), but we also introduce new metrics based on the traditional precision and recall. Secondly, we in- vestigate how two classes of models perform on the task of graded …

Content truncated to 500 words