Using Web-Search Results to Measure Word-Group Similarity

Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 281–288 Manchester, August 2008 Using Web-Search Results to Measure Word-Group Similarity Ann Gledson and John Keane School of Computer Science, University of Manchester Oxford Road, Manchester, UK M13 9PL {ann.gledson,john.keane}@manchester.ac.uk Abstract Semantic relatedness between words is important to many NLP tasks, and nu- merous measures exist which use a vari- ety of resources. Thus far, such work is confined to measuring similarity between two words (or two texts), and only a handful utilize the web as a corpus. This paper introduces a distributional similar- ity measure which uses internet search counts and also extends to calculating the similarity within word-groups. The evaluation results are encouraging: for word-pairs, the correlations with human judgments are comparable with state-of- the-art web-search page-count heuristics. When used to measure similarities within sets of 10 words, the results correlate highly (up to 0.8) with those expected. Relatively little comparison has been made between the results of different search-engines. Here, we compare ex- perimental results from Google, Win- dows Live Search and Yahoo and find noticeable differences. 1 Introduction The propensity of words to appear together in texts, also known as their distributional similarity is an important part of Natural Language Proc- essing (NLP): ‘The need to determine semantic related- ness… between two lexically expressed concepts is a problem that pervades much of [NLP].’ (Budanitsky and Hirst 2006) © 2008. Licensed under the Creative Commons Attri- bution-Noncommercial-Share Alike 3.0 Unported license (http://creativecommons.org/licenses/by-nc- sa/3.0/). Some rights reserved. Such requirements are evident in word sense dis- ambiguation (WSD) (Patwardhan et-al 2003), spelling correction (Budanitsky and Hirst 2006) and Lexical Chaining (Morris and Hirst 1991). As well as measuring the co-occurrence of word-pairs, it is also considered useful to extend these measures to calculate the likelihood of sets of words to appear together. For example, Carac- ciolo et-al (2004) evaluate two established topic area detection (TAD) algorithms and indicate text homogeneity as a document feature affecting the results. Furthermore, Gliozzo et-al (2004) and McCarthy et-al (2007) highlight the impor- tance of topic features for WSD and Navigli and Velardi (2005) report differing WSD results for 3 types of text: unfocussed, mid-technical (eg fi- nance articles) and overly technical (eg interop- erability and computer networks). Measures of word-group similarity can be used to calculate the level of topic cohesion in texts, and the po- tential for this to be used to benefit NLP areas such as WSD and TAD has been indicated in Gledson and Keane (2008). We consider web-searching an important part of measuring similarity, as it provides up-to-date information on word co-occurrence frequencies in the largest available collection of English lan- guage documents. We propose a simple measure that uses internet search counts by measuring the decline in the number of hits as more words (from the word-set to be measured) are appended to a query string using the ‘AND’ operator. To offset against the effect of individual word hit- counts, the above gradient is compared to …

Content truncated to 500 words