Learning Semantics and Selectional Preference of Adjective-Noun Pairs

First Joint Conference on Lexical and Computational Semantics (*SEM), pages 70–74, Montr´eal, Canada, June 7-8, 2012. c⃝2012 Association for Computational Linguistics Learning Semantics and Selectional Preference of Adjective-Noun Pairs Karl Moritz Hermann Department of Computer Science University of Oxford Oxford OX1 3QD, UK karl.moritz.hermann@cs.ox.ac.uk Chris Dyer Language Technologies Institute Carnegie Mellon University Pittsburgh, PA, 15213, USA cdyer@cs.cmu.edu Phil Blunsom Department of Computer Science University of Oxford Oxford OX1 3QD, UK phil.blunsom@cs.ox.ac.uk Stephen Pulman Department of Computer Science University of Oxford Oxford OX1 3QD, UK stephen.pulman@cs.ox.ac.uk Abstract We investigate the semantic relationship be- tween a noun and its adjectival modifiers. We introduce a class of probabilistic mod- els that enable us to to simultaneously cap- ture both the semantic similarity of nouns and modifiers, and adjective-noun selectional preference. Through a combination of novel and existing evaluations we test the degree to which adjective-noun relationships can be cat- egorised. We analyse the effect of lexical con- text on these relationships, and the efficacy of the latent semantic representation for disam- biguating word meaning. 1 Introduction Developing models of the meanings of words and phrases is a key challenge for computational linguis- tics. Distributed representations are useful in captur- ing such meaning for individual words (Sato et al., 2008; Maas and Ng, 2010; Curran, 2005). How- ever, finding a compelling account of semantic com- positionality that utilises such representations has proven more difficult and is an active research topic (Mitchell and Lapata, 2008; Baroni and Zamparelli, 2010; Grefenstette and Sadrzadeh, 2011). It is in this area that our paper makes its contribution. The dominant approaches to distributional se- mantics have relied on relatively simple frequency counting techniques. However, such approaches fail to generalise to the much sparser distributions en- countered when modeling compositional processes and provide no account of selectional preference. We propose a probabilistic model of the semantic representations for nouns and modifiers. The foun- dation of this model is a latent variable representa- tion of noun and adjective semantics together with their compositional probabilities. We employ this formulation to give a dual view of noun-modifier semantics: the induced latent variables provide an explicit account of selectional preference while the marginal distributions of the latent variables for each word implicitly produce a distributed representation. Most related work on selectional preference uses class-based probabilities to approximate (sparse) individual probabilities. Relevant papers include ´O S´eaghdha (2010), who evaluates several topic models adapted to learning selectional preference using co-occurence and Baroni and Zamparelli (2010), who represent nouns as vectors and adjec- tives as matrices, thus treating them as functions over noun meaning. Again, inference is achieved using co-occurrence and dimensionality reduction. 2 Adjective-Noun Model We hypothesize that semantic classes determine the semantic characteristics of nouns and adjectives, and that the distribution of either with respect to other components of the sentences they occur in is also mediated by these classes (i.e., not by the words themselves). We assume that in general nouns select for adjectives,1 and that this selection is dependent on both their latent …

Content truncated to 500 words