A comparison of models of word meaning in context

2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 611–615, Montr´eal, Canada, June 3-8, 2012. c⃝2012 Association for Computational Linguistics A comparison of models of word meaning in context Georgiana Dinu Universität des Saarlandes Saarbrücken, Germany dinu@coli.uni-saarland.de Stefan Thater Universität des Saarlandes Saarbrücken, Germany stth@coli.uni-saarland.de Sören Laue Friedrich-Schiller Universität Jena, Germany soeren.laue@uni-jena.de Abstract This paper compares a number of recently pro- posed models for computing context sensitive word similarity. We clarify the connections between these models, simplify their formula- tion and evaluate them in a unified setting. We show that the models are essentially equivalent if syntactic information is ignored, and that the substantial performance differences previously reported disappear to a large extent when these simplified variants are evaluated under identi- cal conditions. Furthermore, our reformulation allows for the design of a straightforward and fast implementation. 1 Introduction The computation of semantic similarity scores be- tween words is an important sub-task for a variety of NLP applications (Turney and Pantel, 2010). One standard approach is to exploit the so-called distribu- tional hypothesis that similar words tend to appear in similar contexts: Word meaning is represented by the contexts in which a word occurs, and semantic similarity is computed by comparing these contexts in a high-dimensional vector space. Such distributional models of word meaning are attractive because they are simple, have wide cover- age, and can be easily acquired in an unsupervised way. Ambiguity, however, is a fundamental problem: when encountering a word in context, we want a dis- tributional representation which reflects its meaning in this specific context. For instance, while buy and acquire are similar when we consider them in iso- lation, they do not convey the same meaning when acquire occurs in students acquire knowledge. This is particularly difficult for vector space models which compute a single type vector summing up over all occurrences of a word. This vector mixes all of a word’s usages and makes no distinctions between its—potentially very diverse—senses. Several proposals have been made in the recent literature to address this problem. Type-based meth- ods combine the (type) vector of the target with the vectors of the surrounding context words to obtain a disambiguated representation. In recent work, this has been proposed by Mitchell and Lapata (2008), Erk and Padó (2008) and Thater et al. (2010; 2011), which differ in the choice of input vector representa- tion and in the combination operation they propose. A different approach has been taken by Erk and Padó (2010), Reisinger and Mooney (2010) and Reddy et al. (2011), who make use of token vectors for individual occurrences of a word, rather than us- ing the already mixed type vectors. Generally speak- ing, these methods “select” a set of token vectors of the target, which are similar to the current con- text, and use only these to obtain a disambiguated representation. Yet another approach has been taken by Dinu and Lapata (2010), Ó Séaghdha and Korhonen (2011) and Van de Cruys …

Content truncated to 500 words