Investigations on Word Senses and Word Usages

Proceedings of the 47th Annual Meeting of the ACL and the 4th IJCNLP of the AFNLP, pages 10–18, Suntec, Singapore, 2-7 August 2009. c⃝2009 ACL and AFNLP Investigations on Word Senses and Word Usages Katrin Erk University of Texas at Austin katrin.erk@mail.utexas.edu Diana McCarthy University of Sussex dianam@sussex.ac.uk Nicholas Gaylord University of Texas at Austin nlgaylord@mail.utexas.edu Abstract The vast majority of work on word senses has relied on predefined sense invento- ries and an annotation schema where each word instance is tagged with the best fit- ting sense. This paper examines the case for a graded notion of word meaning in two experiments, one which uses WordNet senses in a graded fashion, contrasted with the “winner takes all” annotation, and one which asks annotators to judge the similar- ity of two usages. We find that the graded responses correlate with annotations from previous datasets, but sense assignments are used in a way that weakens the case for clear cut sense boundaries. The responses from both experiments correlate with the overlap of paraphrases from the English lexical substitution task which bodes well for the use of substitutes as a proxy for word sense. This paper also provides two novel datasets which can be used for eval- uating computational systems. 1 Introduction The vast majority of work on word sense tag- ging has assumed that predefined word senses from a dictionary are an adequate proxy for the task, although of course there are issues with this enterprise both in terms of cognitive valid- ity (Hanks, 2000; Kilgarriff, 1997; Kilgarriff, 2006) and adequacy for computational linguis- tics applications (Kilgarriff, 2006). Furthermore, given a predefined list of senses, annotation efforts and computational approaches to word sense dis- ambiguation (WSD) have usually assumed that one best fitting sense should be selected for each us- age. While there is usually some allowance made for multiple senses, this is typically not adopted by annotators or computational systems. Research on the psychology of concepts (Mur- phy, 2002; Hampton, 2007) shows that categories in the human mind are not simply sets with clear- cut boundaries: Some items are perceived as more typical than others (Rosch, 1975; Rosch and Mervis, 1975), and there are borderline cases on which people disagree more often, and on whose categorization they are more likely to change their minds (Hampton, 1979; McCloskey and Glucks- berg, 1978). Word meanings are certainly related to mental concepts (Murphy, 2002). This raises the question of whether there is any such thing as the one appropriate sense for a given occurrence. In this paper we will explore using graded re- sponses for sense tagging within a novel annota- tion paradigm. Modeling the annotation frame- work after psycholinguistic experiments, we do not train annotators to conform to sense distinc- tions; rather we assess individual differences by asking annotators to produce graded ratings in- stead of making a binary choice. We perform two annotation studies. In the first one, referred to as WSsim (Word Sense Similarity), annotators give graded ratings on the …

Content truncated to 500 words