Information Content Measures of Semantic Similarity Perform Better Without Sense-Tagged Text

Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL, pages 329–332, Los Angeles, California, June 2010. c⃝2010 Association for Computational Linguistics Information Content Measures of Semantic Similarity Perform Better Without Sense-Tagged Text Ted Pedersen Department of Computer Science University of Minnesota, Duluth Duluth, MN 55812 tpederse@d.umn.edu http://wn-similarity.sourceforge.net Abstract This paper presents an empirical comparison of similarity measures for pairs of concepts based on Information Content. It shows that using modest amounts of untagged text to de- rive Information Content results in higher cor- relation with human similarity judgments than using the largest available corpus of manually annotated sense–tagged text. 1 Introduction Measures of semantic similarity based on WordNet have been widely used in Natural Language Pro- cessing. These measures rely on the structure of WordNet to produce a numeric score that quantifies the degree to which two concepts (represented by a sense or synset) are similar (or not). In their sim- plest form these measures use path length to identify concepts that are physically close to each other and therefore considered to be more similar than con- cepts that are further apart. While this is a reasonable first approximation to semantic similarity, there are some well known limi- tations. Most significant is that path lengths between very specific concepts imply much smaller distinc- tions in semantic similarity than do comparable path lengths between very general concepts. One pro- posed improvement is to augment concepts in Word- Net with Information Content values derived from sense–tagged corpora or from raw unannotated cor- pora (Resnik, 1995). This paper shows that Information Content mea- sures based on modest amounts of unannotated cor- pora have greater correlation with human similarity judgements than do those based on the largest corpus of sense-tagged text currently available.1 The key to this success is not in the specific type of corpora used, but rather in increasing the number of con- cepts in WordNet that have counts associated with them. These results show that Information Content measures of semantic similarity can be significantly improved without requiring the creation of sense– tagged corpora (which is very expensive). 1.1 Information Content Information Content (IC) is a measure of specificity for a concept. Higher values are associated with more specific concepts (e.g., pitch fork), while those with lower values are more general (e.g., idea). In- formation Content is computed based on frequency counts of concepts as found in a corpus of text. The frequency associated with a concept is incremented in WordNet each time that concept is observed, as are the counts of the ancestor concepts in the Word- Net hierarchy (for nouns and verbs). This is neces- sary because each occurrence of a more specific con- cept also implies the occurrence of the more general ancestor concepts. When a corpus is sense–tagged, mapping occur- rences of a word to a concept is straightforward (since each sense of a word corresponds with a con- cept or synset in WordNet). However, if the text has not been sense–tagged …

Content truncated to 500 words