Detecting Compositionality of Multi-Word Expressions using Nearest Neighbours in Vector Space Models

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1427–1432, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Detecting Compositionality of Multi-Word Expressions using Nearest Neighbours in Vector Space Models Douwe Kiela University of Cambridge Computer Laboratory douwe.kiela@cl.cam.ac.uk Stephen Clark University of Cambridge Computer Laboratory stephen.clark@cl.cam.ac.uk Abstract We present a novel unsupervised approach to detecting the compositionality of multi-word expressions. We compute the compositional- ity of a phrase through substituting the con- stituent words with their “neighbours” in a se- mantic vector space and averaging over the distance between the original phrase and the substituted neighbour phrases. Several meth- ods of obtaining neighbours are presented. The results are compared to existing super- vised results and achieve state-of-the-art per- formance on a verb-object dataset of human compositionality ratings. 1 Introduction Multi-word expressions (MWEs) are defined as “id- iosyncratic interpretations that cross word bound- aries” (Sag et al., 2002). They tend to have a standard syntactic structure but are often semanti- cally non-compositional; i.e. their meaning is not fully determined by their syntactic structure and the meanings of their constituents. A classic example is kick the bucket, which means to die rather than to hit a bucket with the foot. These types of expres- sions account for a large proportion of day-to-day language interactions (Schuler and Joshi, 2011) and present a significant problem for natural language processing systems (Sag et al., 2002). This paper presents a novel unsupervised ap- proach to detecting the compositionality of MWEs, specifically of verb-noun collocations. The idea is that we can recognize compositional phrases by sub- stituting related words for constituent words in the phrase: if the result of a substitution yields a mean- ingful phrase, its individual constituents are likely to contribute toward the overall meaning of the phrase. Conversely, if a substitution yields a non-sensical phrase, its constituents are likely to contribute less or not at all to the overall meaning of the phrase. For the phrase eat her hat, for example, we might consider the following substituted phrases: 1. consume her hat 2. eat her trousers Both phrases are semantically anomalous, implying that eat hat is a highly non-compositional verb-noun collocation. Following a similar procedure for eat apple, however, would not lead to an anomaly: con- sume apple and eat pear are perfectly meaningful, leading us to believe that eat apple is compositional. In the context of distributional models, this idea can be formalised in terms of vector spaces: the average distance between a phrase vector and its substituted phrase vectors is related to its compositionality. Since we are relying on the relative distances of phrases in semantic space, we require a method for computing vectors for phrases. We experi- mented with a number of composition operators from Mitchell and Lapata (2010), in order to com- pose constituent word vectors into phrase vectors. The relation between phrase vectors and substituted phrase vectors is most pronounced in the case of 1427 pointwise multiplication, which has the effect of …

Content truncated to 500 words