SemEval-2013 Task 13: Word Sense Induction for Graded and Non-Graded Senses

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Seventh International Workshop on Semantic Evaluation (SemEval 2013), pages 290–299, Atlanta, Georgia, June 14-15, 2013. c⃝2013 Association for Computational Linguistics SemEval-2013 Task 13: Word Sense Induction for Graded and Non-Graded Senses David Jurgens Dipartimento di Informatica Sapienza Universit`a di Roma jurgens@di.uniroma1.it Ioannis Klapaftis Search Technology Center Europe Microsoft ioannisk@microsoft.com Abstract Most work on word sense disambiguation has assumed that word usages are best labeled with a single sense. However, contextual am- biguity or fine-grained senses can potentially enable multiple sense interpretations of a us- age. We present a new SemEval task for evalu- ating Word Sense Induction and Disambigua- tion systems in a setting where instances may be labeled with multiple senses, weighted by their applicability. Four teams submitted nine systems, which were evaluated in two settings. 1 Introduction Word Sense Disambiguation (WSD) attempts to identify which of a word’s meanings applies in a given context. A long-standing task, WSD is fun- damental to many NLP applications (Navigli, 2009). Typically, each usage of a word is treated as express- ing only a single sense. However, contextual ambi- guity as well as the relatedness of certain meanings can potentially elicit multiple sense interpretations. Recent work has shown that annotators find multi- ple applicable senses in a given target word context when using fine-grained sense inventories such as WordNet (V´eronis, 1998; Murray and Green, 2004; Erk et al., 2009; Passonneau et al., 2012b; Jurgens, 2013; Navigli et al., 2013). Such contexts would be better annotated with multiple sense labels, weight- ing each sense according to its applicability (Erk et al., 2009; Jurgens, 2013), in effect allowing ambigu- ity or multiple interpretations to be explicitly mod- eled. Accordingly, the first goal of this task is to evaluate WSD systems in a setting where instances may be labeled with one or more senses, weighted by their applicability. WSD methods are ultimately defined and poten- tially restricted by their choice in sense inventory; for example, a sense inventory may have insufficient sense-annotated data to build WSD systems for spe- cific types of text (e.g., social media), or the inven- tory may lack domain-specific senses. Word Sense Induction (WSI) has been proposed as a method for overcoming such limitations by learning the senses automatically from text. In essence, a WSI algo- rithm acts as a lexicographer by grouping word us- ages according to their shared meaning. The sec- ond goal of this task is to assess the performance of WSI algorithms when they are able to model multi- ple meanings of a usage with graded senses. Task 12 focuses on disambiguating senses for 50 target lemmas: 20 nouns, 20 verbs, and 10 adjectives (Sec. 2). Since the Task evaluates only unsupervised systems, no training data was provided; however, to enable more comparison, Unsupervised WSD sys- tems were also allowed to participate. Participat- ing systems were evaluated in two settings (Sec. 3), depending on whether they used induced senses or WordNet 3.1 senses for their annotations. …

Content truncated to 500 words