Inducing Latent Semantic Relations for Structured Distributional Semantics

Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 698–708, Dublin, Ireland, August 23-29 2014. Inducing Latent Semantic Relations for Structured Distributional Semantics Sujay Kumar Jauhar Language Technologies Institute School of Computer Science Carnegie Mellon University sjauhar@cs.cmu.edu Eduard Hovy Language Technologies Institute School of Computer Science Carnegie Mellon University hovy@cs.cmu.edu Abstract Structured distributional semantic models aim to improve upon simple vector space models of semantics by hypothesizing that the meaning of a word is captured more effectively through its relational — rather than its raw distributional — signature. In accordance, they extend the vector space paradigm by structuring elements with relational information that decompose distributional signatures over discrete relation dimensions. However, the number and nature of these relations remains an open research question, with most previous work in the literature employing syn- tactic dependencies as surrogates for truly semantic relations. In this paper we propose a novel structured distributional semantic model with latent relation dimensions, and instantiate it using latent relational analysis. Evaluation of our model yields results that significantly outperform several other distributional approaches on two semantic tasks and performs competitively on a third relation classification task. 1 Introduction The distributional hypothesis, articulated by Firth (1957) in the popular dictum “You shall know the word by the company it keeps”, has established itself as one of the most popular models of modern computational semantics. With the rise of massive and easily-accessible digital corpora, computation of co-occurrence statistics has enabled researchers in NLP to build distributional semantic models (DSMs) that have found relevance in many application areas. These include information retrieval (Manning et al., 2008), question answering (Tellex et al., 2003), word-sense disambiguation (McCarthy et al., 2004) and selectional preference modelling (Erk, 2007), to name only a few. The standard DSM framework, which models the semantics of a word by co-occurrence statistics computed over its neighbouring words, has several known short-comings. One severe short-coming derives from the fundamental nature of the vector space model, which characterizes the semantics of a word by a single vector in a high dimensional space (or some lower dimensional embedding thereof). Such a modelling paradigm goes against the grain of the intuition that the semantics of a word is neither unique nor constant. Rather, it is composed of many facets of meaning, and similarity (or dissimilarity) to other words is an outcome of the aggregate harmony (or dissonance) between the individual facets under consideration. For example, a shirt may be similar along one facet to a balloon in that they are both coloured blue, at the same time being similar to a shoe along another facet for both being articles of clothing, while being dissimilar along yet another facet to a t-shirt because one is stitched from linen while the other is made from polyester. Structured distributional semantic models (SDSMs) aim to remedy this fault with DSMs by decompos- ing distributional signatures over discrete relation dimensions, or facets. This leads to a representation that characterizes the semantics of a word …

Content truncated to 500 words