Towards a Formal Distributional Semantics: Simulating Logical Calculi with Tensors

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 1: Proceedings of the Main Conference and the Shared Task, pages 1–10, Atlanta, Georgia, June 13-14, 2013. c⃝2013 Association for Computational Linguistics Towards a Formal Distributional Semantics: Simulating Logical Calculi with Tensors Edward Grefenstette University of Oxford Department of Computer Science Wolfson Building, Parks Road Oxford OX1 3QD, UK edward.grefenstette@cs.ox.ac.uk Abstract The development of compositional distribu- tional models of semantics reconciling the em- pirical aspects of distributional semantics with the compositional aspects of formal seman- tics is a popular topic in the contemporary lit- erature. This paper seeks to bring this rec- onciliation one step further by showing how the mathematical constructs commonly used in compositional distributional models, such as tensors and matrices, can be used to sim- ulate different aspects of predicate logic. This paper discusses how the canonical iso- morphism between tensors and multilinear maps can be exploited to simulate a full-blown quantifier-free predicate calculus using ten- sors. It provides tensor interpretations of the set of logical connectives required to model propositional calculi. It suggests a variant of these tensor calculi capable of modelling quantifiers, using few non-linear operations. It finally discusses the relation between these variants, and how this relation should consti- tute the subject of future work. 1 Introduction The topic of compositional distributional semantics has been growing in popularity over the past few years. This emerging sub-field of natural language semantic modelling seeks to combine two seemingly orthogonal approaches to modelling the meaning of words and sentences, namely formal semantics and distributional semantics. These approaches, summarised in Section 2, dif- fer in that formal semantics, on the one hand, pro- vides a neatly compositional picture of natural lan- guage meaning, reducing sentences to logical rep- resentations; one the other hand, distributional se- mantics accounts for the ever-present ambiguity and polysemy of words of natural language, and pro- vides tractable ways of learning and comparing word meanings based on corpus data. Recent efforts, some of which are briefly re- ported below, have been made to unify both of these approaches to language modelling to pro- duce compositional distributional models of seman- tics, leveraging the learning mechanisms of distri- butional semantics, and providing syntax-sensitive operations for the production of representations of sentence meaning obtained through combination of corpus-inferred word meanings. These efforts have been met with some success in evaluations such as phrase similarity tasks (Mitchell and Lapata, 2008; Mitchell and Lapata, 2009; Grefenstette and Sadrzadeh, 2011; Kartsaklis et al., 2012), sentiment prediction (Socher et al., 2012), and paraphrase de- tection (Blacoe and Lapata, 2012). While these developments are promising with regard to the goal of obtaining learnable-yet- structured sentence-level representations of lan- guage meaning, part of the motivation for unifying formal and distributional models of semantics has been lost. The compositional aspects of formal se- mantics are combined with the corpus-based empir- ical aspects of distributional semantics in such mod- els, yet the logical aspects are not. But it is these logical aspects which are so …

Content truncated to 500 words