A Structured Vector Space Model for Word Meaning in Context

Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 897–906, Honolulu, October 2008. c⃝2008 Association for Computational Linguistics A Structured Vector Space Model for Word Meaning in Context Katrin Erk Department of Linguistics University of Texas at Austin katrin.erk@mail.utexas.edu Sebastian Pad´o Department of Linguistics Stanford University pado@stanford.edu Abstract We address the task of computing vector space representations for the meaning of word oc- currences, which can vary widely according to context. This task is a crucial step towards a robust, vector-based compositional account of sentence meaning. We argue that existing mod- els for this task do not take syntactic structure sufficiently into account. We present a novel structured vector space model that addresses these issues by incorpo- rating the selectional preferences for words’ argument positions. This makes it possible to integrate syntax into the computation of word meaning in context. In addition, the model per- forms at and above the state of the art for mod- eling the contextual adequacy of paraphrases. 1 Introduction Semantic spaces are a popular framework for the rep- resentation of word meaning, encoding the meaning of lemmas as high-dimensional vectors. In the de- fault case, the components of these vectors measure the co-occurrence of the lemma with context features over a large corpus. These vectors are able to pro- vide a robust model of semantic similarity that has been used in NLP (Salton et al., 1975; McCarthy and Carroll, 2003; Manning et al., 2008) and to model experimental results in cognitive science (Landauer and Dumais, 1997; McDonald and Ramscar, 2001). Semantic spaces are attractive because they provide a model of word meaning that is independent of dictio- nary senses and their much-discussed problems (Kil- garriff, 1997; McCarthy and Navigli, 2007). In a default semantic space as described above, each vector represents one lemma, averaging over all its possible usages (Landauer and Dumais, 1997; Lund and Burgess, 1996). Since the meaning of words can vary substantially between occurrences (e.g., for polysemous words), the next necessary step is to characterize the meaning of individual words in context. There have been several approaches in the liter- ature (Smolensky, 1990; Sch¨utze, 1998; Kintsch, 2001; McDonald and Brew, 2004; Mitchell and La- pata, 2008) that compute meaning in context from lemma vectors. Most of these studies phrase the prob- lem as one of vector composition: The meaning of a target occurrence a in context b is a single new vector c that is a function (for example, the centroid) of the vectors: c = a ⊙ b. The context b can consist of as little as one word, as shown in Example (1). In (1a), the meaning of catch combined with ball is similar to grab, while in (1b), combined with disease, it can be paraphrased by contract. Conversely, verbs can influence the in- terpretation of nouns: In (1a), ball is understood as a spherical object, and in (1c) as a dancing event. (1) a. catch a ball b. catch a disease c. attend a ball …

Content truncated to 500 words