First Order vs. Higher Order Modification in Distributional Semantics

Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1223–1233, Jeju Island, Korea, 12–14 July 2012. c⃝2012 Association for Computational Linguistics First-order vs. higher-order modification in distributional semantics Gemma Boleda Linguistics Department University of Texas at Austin gemma.boleda@utcompling.com Eva Maria Vecchi Center for Mind/Brain Sciences University of Trento evamaria.vecchi@unitn.it Miquel Cornudella and Louise McNally Departament de Traducci´o i Ci`encies del Llenguatge Universitat Pompeu Fabra miquel.cornudellagaya@gmail.com, louise.mcnally@upf.edu Abstract Adjectival modification, particularly by ex- pressions that have been treated as higher- order modifiers in the formal semantics tradi- tion, raises interesting challenges for semantic composition in distributional semantic mod- els. We contrast three types of adjectival mod- ifiers – intersectively used color terms (as in white towel, clearly first-order), subsectively used color terms (white wine, which have been modeled as both first- and higher-order), and intensional adjectives (former bassist, clearly higher-order) – and test the ability of different composition strategies to model their behav- ior. In addition to opening up a new empir- ical domain for research on distributional se- mantics, our observations concerning the at- tested vectors for the different types of adjec- tives, the nouns they modify, and the resulting noun phrases yield insights into modification that have been little evident in the formal se- mantics literature to date. 1 Introduction One of the most appealing aspects of so-called dis- tributional semantic models (see Turney and Pan- tel (2010) for a recent overview) is that they af- ford some hope for a non-trivial, computationally tractable treatment of the context dependence of lex- ical meaning that might also approximate in inter- esting ways the psychological representation of that meaning (Andrews et al., 2009). However, in or- der to have a complete theory of natural language meaning, these models must be supplied with or connected to a compositional semantics; otherwise, we will have no account of the recursive potential that natural language affords for the construction of novel complex contents. In the last 4-5 years, researchers have begun to introduce compositional operations on distribu- tional semantic representations, for instance to com- bine verbs with their arguments or adjectives with nouns (Erk and Pad´o, 2008; Mitchell and Lapata, 2010; Baroni and Zamparelli, 2010; Grefenstette and Sadrzadeh, 2011; Socher et al., 2011)1. Al- though the proposed operations have shown vary- ing degrees of success in a number of tasks such as detecting phrase similarity and paraphrasing, it re- mains unclear to what extent they can account for the full range of meaning composition phenomena found in natural language. Higher-order modifica- tion (that is, modification that cannot obviously be modeled as property intersection, in contrast to first- order modification, which can) presents one such challenge, as we will detail in the next section. The goal of this paper is twofold. First, we exam- ine how the properties of different types of adjecti- val modifiers, both in isolation and in combination with nouns, are represented in distributional mod- els. We take as a case study three …

Content truncated to 500 words