A Tensor-based Factorization Model of Semantic Compositionality

Proceedings of NAACL-HLT 2013, pages 1142–1151, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics A Tensor-based Factorization Model of Semantic Compositionality Tim Van de Cruys IRIT – UMR 5505 CNRS Toulouse, France tim.vandecruys@irit.fr Thierry Poibeau⋆ LaTTiCe – UMR 8094 CNRS & ENS Paris, France thierry.poibeau@ens.fr Anna Korhonen Computer Laboratory & DTAL⋆ University of Cambridge United Kingdom anna.korhonen@cl.cam.ac.uk Abstract In this paper, we present a novel method for the computation of compositionality within a distri- butional framework. The key idea is that com- positionality is modeled as a multi-way interac- tion between latent factors, which are automat- ically constructed from corpus data. We use our method to model the composition of sub- ject verb object triples. The method consists of two steps. First, we compute a latent factor model for nouns from standard co-occurrence data. Next, the latent factors are used to induce a latent model of three-way subject verb object interactions. Our model has been evaluated on a similarity task for transitive phrases, in which it exceeds the state of the art. 1 Introduction In the course of the last two decades, significant progress has been made with regard to the automatic extraction of lexical semantic knowledge from large- scale text corpora. Most work relies on the distribu- tional hypothesis of meaning (Harris, 1954), which states that words that appear within the same contexts tend to be semantically similar. A large number of researchers have taken this dictum to heart, giving rise to a plethora of algorithms that try to capture the semantics of words by looking at their distribu- tion in text. Up till now, however, most work on the automatic acquisition of semantics only deals with individual words. The modeling of meaning beyond the level of individual words – i.e. the combination of words into larger units – is to a large degree left unexplored. The principle of compositionality, often attributed to Frege, is the principle that states that the meaning of a complex expression is a function of the meaning of its parts and the way those parts are (syntactically) combined (Frege, 1892). It is the fundamental prin- ciple that allows language users to understand the meaning of sentences they have never heard before, by constructing the meaning of the complex expres- sion from the meanings of the individual words. Re- cently, a number of researchers have tried to reconcile the framework of distributional semantics with the principle of compositionality (Mitchell and Lapata, 2008; Baroni and Zamparelli, 2010; Coecke et al., 2010; Socher et al., 2012). However, the absolute gains of the systems remain a bit unclear, and a sim- ple method of composition – vector multiplication – often seems to produce the best results (Blacoe and Lapata, 2012). In this paper, we present a novel method for the joint composition of a verb with its subject and di- rect object. The key idea is that compositionality is modeled as a multi-way interaction between latent factors, which are automatically constructed from corpus data. In order …

Content truncated to 500 words