Word Meaning in Context: A Simple and Effective Vector Model

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 1134–1143, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Word Meaning in Context: A Simple and Effective Vector Model Stefan Thater and Hagen Fürstenau and Manfred Pinkal Department of Computational Linguistics Saarland University {stth, hagenf, pinkal}@coli.uni-saarland.de Abstract We present a model that represents word meaning in context by vectors which are modified according to the words in the tar- get’s syntactic context. Contextualization of a vector is realized by reweighting its components, based on distributional infor- mation about the context words. Evalua- tion on a paraphrase ranking task derived from the SemEval 2007 Lexical Substi- tution Task shows that our model outper- forms all previous models on this task. We show that our model supports a wider range of applications by evaluating it on a word sense disambiguation task. Results show that our model achieves state-of-the-art per- formance. 1 Introduction Distributional vector-space models of word mean- ing have proven helpful for a number of basic natu- ral language processing tasks, such as word sense discrimination (Schütze, 1998) and disambiguation (McCarthy et al., 2004), or modeling of selectional preferences (Erk, 2007), and have been success- fully used in a variety of applications like informa- tion retrieval (Manning et al., 2008) or question answering (Tellex et al., 2003). Standard distri- butional models of meaning are attractive because they are simple, have wide coverage, and, in par- ticular, can be acquired using unsupervised meth- ods at virtually no cost. Vector-space models of meaning lend themselves as a basis for determining a soft and gradual concept of semantic similarity (e.g., through the cosine measure), which does not rely on a fixed set of dictionary senses with their well-known problems (Kilgarriff, 1997). The sensitivity of word meaning to the context of use, however, poses a major challenge for dis- tributional semantics. Meaning vectors are based on co-occurrence counts for words across all word senses and usages. This means that, for instance, any occurrence of the verb charge, such as in the expressions charge a fee or charge a battery, is as- signed the same vector representation, ignoring the difference of word sense. On the other hand, the fact that charge and impose are near-synonyms in charge/impose a fee will not be properly reflected in their respective meaning vectors, since the for- mer, but not the latter, includes (context words reflecting) the “supply electricity” sense of charge. The problem of modeling context-sensitivity in a distributional framework has first been addressed in the seminal paper of Schütze (1998), who uses second-order bag-of-words vectors for the task of word sense discrimination. Recently, the issue has been taken up by several approaches that in- clude some kind of syntactic information, in part under the heading of “distributional composition- ality” (Mitchell and Lapata, 2008; Erk and Padó, 2008), in part as “syntax-sensitive contextualiza- tion” (Thater et al., 2010). These approaches have in common that the contextual influence on the meaning of a target word w is …

Content truncated to 500 words