Duluth : Measuring Degrees of Relational Similarity with the Gloss Vector Measure of Semantic Relatedness

First Joint Conference on Lexical and Computational Semantics (*SEM), pages 497–501, Montr´eal, Canada, June 7-8, 2012. c⃝2012 Association for Computational Linguistics Duluth : Measuring Degrees of Relational Similarity with the Gloss Vector Measure of Semantic Relatedness Ted Pedersen Department of Computer Science University of Minnesota Duluth, MN 55812 USA tpederse@d.umn.edu Abstract This paper describes the Duluth systems that participated in Task 2 of SemEval–2012. These systems were unsupervised and relied on variations of the Gloss Vector measure found in the freely available software pack- age WordNet::Similarity. This method was moderately successful for the Class-Inclusion, Similar, Contrast, and Non-Attribute cate- gories of semantic relations, but mimicked a random baseline for the other six categories. 1 Introduction This paper describes the Duluth systems that par- ticipated in Task 2 of SemEval–2012, Measuring the Degree of Relational Similarity (Jurgens et al., 2012). The goal of the task was to rank sets of word pairs according to the degree to which they represented an underlying category of semantic re- lation. A highly ranked pair would be considered a good or prototypical example of the relation. For example, given the relation Y functions as an X the pair weapon:knife (X:Y) would likely be considered more representative of that relation than would be tool:spoon. The task included word pairs from 10 different categories of relational similarity, each with a num- ber of subcategories. In total the evaluation data consisted of 69 files, each containing a set of ap- proximately 40 word pairs. While training examples were also provided, these were not used by the Du- luth systems. The system–generated rankings were compared with gold standard data created via Ama- zon Mechanical Turk. The Duluth systems relied on the Gloss Vec- tor measure of semantic relatedness (Patwardhan and Pedersen, 2006) as implemented in Word- Net::Similarity (Pedersen et al., 2004)1. This quanti- fies the degree of semantic relatedness between two word senses. It does not, however, discover or in- dicate the nature of the relation between the words. When given two words as input (as was the case in this task), it measures the relatedness of all possi- ble combinations of word senses associated with this pair and reports the highest resulting score. Note that throughout this paper we use word and word sense somewhat interchangeably. In general it may be assumed that the term word or examples of words refers to a word sense. A key characteristic of this task was that the word pairs in each of the 69 sets were scored assuming a particular specified underlying semantic relation. Given this, the limitation that the Gloss Vector mea- sure does not discover the nature of relations was less of a concern, and led to the hypothesis that a word pair that was highly related would also be a prototypical example of the underlying category of semantic relation. Unfortunately the results from this task do not generally support this hypothesis, although for a few categories at least it appears to have some validity. This paper continues with …

Content truncated to 500 words