Concrete Models and Empirical Evaluations for the Categorical Compositional Distributional Model of Meaning

Concrete Models and Empirical Evaluations for the Categorical Compositional Distributional Model of Meaning Edward Grefenstette∗† Google DeepMind Mehrnoosh Sadrzadeh∗∗† Queen Mary University of London Modeling compositional meaning for sentences using empirical distributional methods has been a challenge for computational linguists. The categorical model of Clark, Coecke, and Sadrzadeh (2008) and Coecke, Sadrzadeh, and Clark (2010) provides a solution by unifying a categorial grammar and a distributional model of meaning. It takes into account syntactic relations during semantic vector composition operations. But the setting is abstract: It has not been evaluated on empirical data and applied to any language tasks. We generate concrete models for this setting by developing algorithms to construct tensors and linear maps and instantiate the abstract parameters using empirical data. We then evaluate our concrete models against several experiments, both existing and new, based on measuring how well models align with human judgments in a paraphrase detection task. Our results show the implementation of this general abstract framework to perform on par with or outperform other leading models in these experiments.1 1. Introduction The distributional approach to the semantic modeling of natural language, inspired by the notion—presented by Firth (1957) and Harris (1968)—that the meaning of a word is tightly related to its context of use, has grown in popularity as a method of semantic representation. It draws from the frequent use of vector-based document models in information retrieval, modeling the meaning of words as vectors based on the distribution of co-occurring terms within the context of a word. Using various vector similarity metrics as a measure of semantic similarity, these distributional semantic models (DSMs) are used for a variety of NLP tasks, from ∗ DeepMind Technologies Ltd, 5 New Street Square, London EC4A 3 TW. E-mail: etg@google.com. ∗∗ School of Electronic Engineering and Computer Science, Queen Mary University of London, Mile End Road, London E1 4NS, United Kingdom. E-mail: mehrnoosh.sadrzadeh@qmul.ac.uk. † The work described in this article was performed while the authors were at the University of Oxford. 1 Support from EPSRC grant EP/J002607/1 is acknowledged. Submission received: 26 September 2012; revised submission received: 31 October 2013; accepted for publication: 5 April 2014. doi:10.1162/COLI a 00209 © 2015 Association for Computational Linguistics Computational Linguistics Volume 41, Number 1 automated thesaurus building (Grefenstette 1994; Curran 2004) to automated essay marking (Landauer and Dumais 1997). The broader connection to information retrieval and its applications is also discussed by Manning, Raghavan, and Sch¨utze (2011). The success of DSMs in essentially word-based tasks such as thesaurus extraction and con- struction (Grefenstette 1994; Curran 2004) invites an investigation into how DSMs can be applied to NLP and information retrieval (IR) tasks revolving around larger units of text, using semantic representations for phrases, sentences, or documents, constructed from lemma vectors. However, the problem of compositionality in DSMs—of how to go from word to sentence and beyond—has proved to be non-trivial. A new framework, which we refer to as DisCoCat, initially presented in Clark, Coecke, and Sadrzadeh (2008) and Coecke, Sadrzadeh, and Clark …

Content truncated to 500 words