Dynamic and Static Prototype Vectors for Semantic Composition

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 705–713, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Dynamic and Static Prototype Vectors for Semantic Composition Siva Reddy University of York, UK siva@cs.york.ac.uk Ioannis P. Klapaftis University of York, UK giannis@cs.york.ac.uk Diana McCarthy Lexical Computing Ltd, UK diana@dianamccarthy.co.uk Suresh Manandhar University of York, UK suresh@cs.york.ac.uk Abstract Compositional Distributional Semantic methods model the distributional behav- ior of a compound word by exploiting the distributional behavior of its constituent words. In this setting, a constituent word is typically represented by a feature vec- tor conflating all the senses of that word. However, not all the senses of a constituent word are relevant when composing the se- mantics of the compound. In this paper, we present two different methods for se- lecting the relevant senses of constituent words. The first one is based on Word Sense Induction and creates a static multi prototype vectors representing the senses of a constituent word. The second creates a single dynamic prototype vector for each constituent word based on the distribu- tional properties of the other constituents in the compound. We use these proto- type vectors for composing the seman- tics of noun-noun compounds and evalu- ate on a compositionality-based similar- ity task. Our results show that: (1) se- lecting relevant senses of the constituent words leads to a better semantic compo- sition of the compound, and (2) dynamic prototypes perform better than static pro- totypes. 1 Introduction Vector Space Models of lexical semantics have become a standard framework for representing a word’s meaning. Typically these methods (Sch¨utze, 1998; Pado and Lapata, 2007; Erk and Pad´o, 2008) utilize a bag-of-words model or syntactic dependencies such as subject/verb, ob- ject/verb relations, so as to extract the features which serve as the dimensions of the vector space. Each word is then represented as a vector of the extracted features, where the frequency of co- occurrence of the word with each feature is used to calculate the vector component asociated with that feature. Figure 1 provides an example of two nouns assuming a bag-of-words model. vector dimensions animal buy apartment price rent kill house ⟨ 30 60 90 55 45 10 ⟩ hunting ⟨ 90 15 12 20 33 90 ⟩ Figure 1: A hypothetical vector space model. Compositional Distributional Semantic meth- ods formalise the meaning of a phrase by ap- plying a vector composition function on the vectors associated with its constituent words (Mitchell and Lapata, 2008; Widdows, 2008). For example, the result of vector addition to compose the semantics of house hunting from the vectors house and hunting is the vector ⟨120, 75, 102, 75, 78, 100⟩. As can be observed the resulting vector does not reflect the correct meaning of the compound house hunting due to the presence of irrelevant co-occurrences such as animal or kill. These co- occurrences are relevant to one sense of hunting, i.e. (the activity of hunting animals), but not to the sense of hunting meant in house …

Content truncated to 500 words