A Quantum-Theoretic Approach to Distributional Semantics

Proceedings of NAACL-HLT 2013, pages 847–857, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics A Quantum-Theoretic Approach to Distributional Semantics William Blacoe1, Elham Kashefi2, Mirella Lapata1 1Institute for Language, Cognition and Computation, 2Laboratory for Foundations of Computer Science School of Informatics, University of Edinburgh, 10 Crichton Street, Edinburgh EH8 9AB w.b.blacoe@sms.ed.ac.uk, ekashefi@inf.ed.ac.uk, mlap@inf.ed.ac.uk Abstract In this paper we explore the potential of quantum theory as a formal framework for capturing lexical meaning. We present a novel semantic space model that is syntacti- cally aware, takes word order into account, and features key quantum aspects such as superposition and entanglement. We define a dependency-based Hilbert space and show how to represent the meaning of words by den- sity matrices that encode dependency neigh- borhoods. Experiments on word similarity and association reveal that our model achieves results competitive with a variety of classical models. 1 Introduction The fields of cognitive science and natural language processing have recently produced an ensemble of semantic models which have an impressive track record of replicating human behavior and enabling real-world applications. Examples include simula- tions of word association (Denhi`ere and Lemaire, 2004; Griffiths et al., 2007), semantic priming (Lund and Burgess, 1996; Landauer and Dumais, 1997; Griffiths et al., 2007), categorization (Laham, 2000), numerous studies of lexicon acquisition (Grefen- stette, 1994; Lin, 1998), word sense discrimination (Sch¨utze, 1998), and paraphrase recognition (Socher et al., 2011). The term “semantic” derives from the intuition that words seen in the context of a given word contribute to its meaning (Firth, 1957). Al- though the specific details of the individual models differ, they all process a corpus of text as input and represent words (or concepts) in a (reduced) high- dimensional space. In this paper, we explore the potential of quan- tum theory as a formal framework for capturing lex- ical meaning and modeling semantic processes such as word similarity and association (see Section 6 for an overview of related research in this area). We use the term quantum theory to refer to the ab- stract mathematical foundation of quantum mechan- ics which is not specifically tied to physics (Hughes, 1989; Isham, 1989). Quantum theory is in prin- ciple applicable in any discipline where there is a need to formalize uncertainty. Indeed, researchers have been pursuing applications in areas as diverse as economics (Baaquie, 2004), information theory (Nielsen and Chuang, 2010), psychology (Khren- nikov, 2010; Pothos and Busemeyer, 2012), and cog- nitive science (Busemeyer and Bruza, 2012; Aerts, 2009; Bruza et al., 2008). But what are the features of quantum theory which make it a promising frame- work for modeling meaning? Superposition, entanglement, incompatibility, and interference are all related aspects of quantum theory, which endow it with a unique character.1 Su- perposition is a way of modeling uncertainty, more so than in classical probability theory. It contains in- formation about the potentialities of a system’s state. An electron whose location in an atom is uncertain can be modeled as being in a superposition of loca- tions. Analogously, words …

Content truncated to 500 words