Characterising Measures of Lexical Distributional Similarity

Characterising Measures of Lexical Distributional Similarity Julie Weeds, David Weir and Diana McCarthy Department of Informatics University of Sussex Brighton, BN1 9QH, UK {juliewe, davidw,dianam}@sussex.ac.uk Abstract This work investigates the variation in a word’s dis- tributionally nearest neighbours with respect to the similarity measure used. We identify one type of variation as being the relative frequency of the neigh- bour words with respect to the frequency of the tar- get word. We then demonstrate a three-way connec- tion between relative frequency of similar words, a concept of distributional gnerality and the seman- tic relation of hyponymy. Finally, we consider the impact that this has on one application of distribu- tional similarity methods (judging the composition- ality of collocations). 1 Introduction Over recent years, many Natural Language Pro- cessing (NLP) techniques have been developed that might benefit from knowledge of distribu- tionally similar words, i.e., words that occur in similar contexts. For example, the sparse data problem can make it difficult to construct lan- guage models which predict combinations of lex- ical events. Similarity-based smoothing (Brown et al., 1992; Dagan et al., 1999) is an intuitively appealing approach to this problem where prob- abilities of unseen co-occurrences are estimated from probabilities of seen co-occurrences of dis- tributionally similar events. Other potential applications apply the hy- pothesised relationship (Harris, 1968) between distributional similarity and semantic similar- ity; i.e., similarity in the meaning of words can be predicted from their distributional similarity. One advantage of automatically generated the- sauruses (Grefenstette, 1994; Lin, 1998; Curran and Moens, 2002) over large-scale manually cre- ated thesauruses such as WordNet (Fellbaum, 1998) is that they might be tailored to a partic- ular genre or domain. However, due to the lack of a tight defini- tion for the concept of distributional similarity and the broad range of potential applications, a large number of measures of distributional sim- ilarity have been proposed or adopted (see Sec- tion 2). Previous work on the evaluation of dis- tributional similarity methods tends to either compare sets of distributionally similar words to a manually created semantic resource (Lin, 1998; Curran and Moens, 2002) or be oriented towards a particular task such as language mod- elling (Dagan et al., 1999; Lee, 1999). The first approach is not ideal since it assumes that the goal of distributional similarity methods is to predict semantic similarity and that the seman- tic resource used is a valid gold standard. Fur- ther, the second approach is clearly advanta- geous when one wishes to apply distributional similarity methods in a particular application area. However, it is not at all obvious that one universally best measure exists for all applica- tions (Weeds and Weir, 2003). Thus, applying a distributional similarity technique to a new ap- plication necessitates evaluating a large number of distributional similarity measures in addition to evaluating the new model or algorithm. We propose a shift in focus from attempting to discover the overall best distributional sim- ilarity measure to analysing the statistical and linguistic properties of sets of …

Content truncated to 500 words