Multi-Prototype Vector-Space Models of Word Meaning

Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL, pages 109–117, Los Angeles, California, June 2010. c⃝2010 Association for Computational Linguistics Multi-Prototype Vector-Space Models of Word Meaning Joseph Reisinger Department of Computer Science The University of Texas at Austin 1 University Station C0500 Austin, TX 78712-0233 joeraii@cs.utexas.edu Raymond J. Mooney Department of Computer Science The University of Texas at Austin 1 University Station C0500 Austin, TX 78712-0233 mooney@cs.utexas.edu Abstract Current vector-space models of lexical seman- tics create a single “prototype” vector to rep- resent the meaning of a word. However, due to lexical ambiguity, encoding word mean- ing with a single vector is problematic. This paper presents a method that uses cluster- ing to produce multiple “sense-specific” vec- tors for each word. This approach provides a context-dependent vector representation of word meaning that naturally accommodates homonymy and polysemy. Experimental com- parisons to human judgements of semantic similarity for both isolated words as well as words in sentential contexts demonstrate the superiority of this approach over both proto- type and exemplar based vector-space models. 1 Introduction Automatically judging the degree of semantic sim- ilarity between words is an important task useful in text classification (Baker and McCallum, 1998), information retrieval (Sanderson, 1994), textual en- tailment, and other language processing tasks. The standard empirical approach to this task exploits the distributional hypothesis, i.e. that similar words ap- pear in similar contexts (Curran and Moens, 2002; Lin and Pantel, 2002; Pereira et al., 1993). Tra- ditionally, word types are represented by a sin- gle vector of contextual features derived from co- occurrence information, and semantic similarity is computed using some measure of vector distance (Lee, 1999; Lowe, 2001). However, due to homonymy and polysemy, cap- turing the semantics of a word with a single vector is problematic. For example, the word club is similar to both bat and association, which are not at all simi- lar to each other. Word meaning violates the triangle inequality when viewed at the level of word types, posing a problem for vector-space models (Tver- sky and Gati, 1982). A single “prototype” vector is simply incapable of capturing phenomena such as homonymy and polysemy. Also, most vector-space models are context independent, while the meaning of a word clearly depends on context. The word club in “The caveman picked up the club” is similar to bat in “John hit the robber with a bat,” but not in “The bat flew out of the cave.” We present a new resource-lean vector-space model that represents a word’s meaning by a set of distinct “sense specific” vectors. The similarity of two isolated words A and B is defined as the mini- mum distance between one of A’s vectors and one of B’s vectors. In addition, a context-dependent mean- ing for a word is determined by choosing one of the vectors in its set based on minimizing the distance to the vector representing the current context. Con- sequently, the model supports judging the similarity of both words …

Content truncated to 500 words