Measuring Distributional Similarity in Context

Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1162–1172, MIT, Massachusetts, USA, 9-11 October 2010. c⃝2010 Association for Computational Linguistics Measuring Distributional Similarity in Context Georgiana Dinu Department of Computational Linguistics Saarland University Saarbr¨ucken, Germany dinu@coli.uni-sb.de Mirella Lapata School of Informatics University of Edinburgh Edinburgh, UK mlap@inf.ed.ac.uk Abstract The computation of meaning similarity as operationalized by vector-based models has found widespread use in many tasks ranging from the acquisition of synonyms and para- phrases to word sense disambiguation and tex- tual entailment. Vector-based models are typ- ically directed at representing words in isola- tion and thus best suited for measuring simi- larity out of context. In his paper we propose a probabilistic framework for measuring sim- ilarity in context. Central to our approach is the intuition that word meaning is represented as a probability distribution over a set of la- tent senses and is modulated by context. Ex- perimental results on lexical substitution and word similarity show that our algorithm out- performs previously proposed models. 1 Introduction The computation of meaning similarity as op- erationalized by vector-based models has found widespread use in many tasks within natural lan- guage processing (NLP). These range from the ac- quisition of synonyms (Grefenstette, 1994; Lin, 1998) and paraphrases (Lin and Pantel, 2001) to word sense disambiguation (Schuetze, 1998), tex- tual entailment (Clarke, 2009), and notably informa- tion retrieval (Salton et al., 1975). The popularity of vector-based models lies in their unsupervised nature and ease of computation. In their simplest incarnation, these models repre- sent the meaning of each word as a point in a high-dimensional space, where each component cor- responds to some co-occurring contextual element (Landauer and Dumais, 1997; McDonald, 2000; Lund and Burgess, 1996). The advantage of taking such a geometric approach is that the similarity of word meanings can be easily quantified by measur- ing their distance in the vector space, or the cosine of the angle between them. Vector-based models do not explicitly identify the different senses of words and consequently repre- sent their meaning invariably (i.e., irrespective of co- occurring context). Consider for example the adjec- tive heavy which we may associate with the gen- eral meaning of “dense” or “massive”. However, when attested in context, heavy may refer to an over- weight person (e.g., She is short and heavy but she has a heart of gold.) or an excessive cannabis user (e.g., Some heavy users develop a psychological de- pendence on cannabis.). Recent work addresses this issue indirectly with the development of specialized models that repre- sent word meaning in context (Mitchell and Lap- ata, 2008; Erk and Pad´o, 2008; Thater et al., 2009). These methods first extract typical co-occurrence vectors representing a mixture of senses and then use vector operations to either obtain contextualized rep- resentations of a target word (Erk and Pad´o, 2008) or a representation for a set of words (Mitchell and Lapata, 2009). In this paper we propose a probabilistic frame- work for representing word meaning and measuring …

Content truncated to 500 words