Distributional vectors encode referential attributes

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 12–21, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Distributional vectors encode referential attributes Abhijeet Gupta∗ Gemma Boleda† Marco Baroni† Sebastian Pad´o∗ ∗Institut f¨ur Maschinelle Sprachverarbeitung Universit¨at Stuttgart †Center for Mind/Brain Sciences University of Trento Abstract Distributional methods have proven to ex- cel at capturing fuzzy, graded aspects of meaning (Italy is more similar to Spain than to Germany). In contrast, it is diffi- cult to extract the values of more specific attributes of word referents from distribu- tional representations, attributes of the kind typically found in structured knowledge bases (Italy has 60 million inhabitants). In this paper, we pursue the hypothesis that distributional vectors also implicitly en- code referential attributes. We show that a standard supervised regres- sion model is in fact sufficient to retrieve such attributes to a reasonable degree of ac- curacy: When evaluated on the prediction of both categorical and numeric attributes of countries and cities, the model consis- tently reduces baseline error by 30%, and is not far from the upper bound. Further anal- ysis suggests that our model is able to “ob- jectify” distributional representations for entities, anchoring them more firmly in the external world in measurable ways. 1 Introduction Distributional models induce vector-based seman- tic representations of words from their contextual distributions in corpora, exploiting the observation that words with related meanings tend to occur in similar linguistic contexts (Turney and Pantel, 2010; Erk, 2012). Since the approach only requires raw text as input, it can be used to harvest word representations on a very large scale. By encoding the rich knowledge that is present in text, these representations are able to capture many aspects of word meaning. Moreover, approximating se- mantic similarity by graded geometric distance in a vector space is an effective strategy to address the many linguistic phenomena that are better charac- terized in gradient rather than discrete terms, such as synonymy, selectional preferences, and semantic priming (Baroni and Lenci, 2010; Erk et al., 2010; Pad´o and Lapata, 2007, among others). However, not all aspects of human semantic knowledge are satisfactorily captured in terms of fuzzy relations and graded similarity. In particular, our knowledge of the meaning of words denoting specific entities involves a number of “hard facts” about the referents they denote that are best for- malized as attribute-value pairs, of the sort that are stored in manually-curated knowledge bases, such as FreeBase or Wikidata.1 While distributional vec- tors can capture the useful fact that, say, Italy is in many ways more similar to Spain than to Germany, as humans we also know (or we can easily look up) a set of objective facts about Italy, such as what is its capital, its area, its official language and GDP, that are difficult to express in the language of vector algebra and geometry. In this paper, we explore the hypothesis that dis- tributional vectors implicitly encode such attributes of referential entities, which we will call …

Content truncated to 500 words