Nouns are Vectors, Adjectives are Matrices: Representing Adjective-Noun Constructions in Semantic Space

Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1183–1193, MIT, Massachusetts, USA, 9-11 October 2010. c⃝2010 Association for Computational Linguistics Nouns are vectors, adjectives are matrices: Representing adjective-noun constructions in semantic space Marco Baroni and Roberto Zamparelli Center for Mind/Brain Sciences, University of Trento Rovereto (TN), Italy {marco.baroni,roberto.zamparelli}@unitn.it Abstract We propose an approach to adjective-noun composition (AN) for corpus-based distribu- tional semantics that, building on insights from theoretical linguistics, represents nouns as vectors and adjectives as data-induced (lin- ear) functions (encoded as matrices) over nominal vectors. Our model significantly out- performs the rivals on the task of reconstruct- ing AN vectors not seen in training. A small post-hoc analysis further suggests that, when the model-generated AN vector is not simi- lar to the corpus-observed AN vector, this is due to anomalies in the latter. We show more- over that our approach provides two novel ways to represent adjective meanings, alter- native to its representation via corpus-based co-occurrence vectors, both outperforming the latter in an adjective clustering task. 1 Introduction An influential approach for representing the mean- ing of a word in NLP is to treat it as a vector that codes the pattern of co-occurrence of that word with other expressions in a large corpus of language (Sahlgren, 2006; Turney and Pantel, 2010). This approach to semantics (sometimes called distribu- tional semantics) naturally captures word cluster- ing, scales well to large lexicons and doesn’t re- quire words to be manually disambiguated (Sch¨utze, 1997). However, until recently it has been limited to the level of content words (nouns, adjectives, verbs), and it hasn’t tackled in a general way compositional- ity (Frege, 1892; Partee, 2004), that crucial property of natural language which allows speakers to de- rive the meaning of a complex linguistic constituent from the meaning of its immediate syntactic subcon- stituents. Formal semantics (FS), the research program stemming from Montague (1970b; 1973), has oppo- site strengths and weaknesses. Its core semantic no- tion is the sentence, not the word; at the lexical level, it focuses on the meaning of function words; one of its main goals is to formulate recursive composi- tional rules that derive the quantificational properties of complex sentences and their antecedent-pronoun dependencies. Given its focus on quantification, FS treats the meanings of nouns and verbs as pure extensions: nouns and (intransitive) verbs are properties, and thus denote sets of individuals. Adjectives are also often assumed to denote properties: in this view redadj would be the set of ‘entities which are red’, plasticadj, the set of ‘objects made of plastic’, and so forth. In the simplest case, the meaning of an at- tributive adjective-noun (AN) constituent can be ob- tained as the intersection of the adjective and noun extensions A∩N: [ red car ] = {... red objects... } ∩ {... cars... } However, the intersective method of combination is well-known to fail in many cases (Kamp, 1975; Montague, 1970a; Siegel, 1976): for instance, a fake gun is not a gun. Even for …

Content truncated to 500 words