Typology of Adjectives Benchmark for Compositional Distributional Models

Typology of Adjectives Benchmark for Compositional Distributional Models Daria Ryzhova1, Maria Kyuseva1, Denis Paperno2 1NRU Higher School of Economics, 2University of Trento 1School of Linguistics – Staraya Basmannaya Ulitsa 21/4, 105066 Moscow, Russia; 2CIMeC – Center for Mind/Brain Sciences Palazzo Fedrigotti – corso Bettini 31, 38068 Rovereto (TN), Italy daria.ryzhova@mail.ru, mkyuseva@gmail.com, denis.paperno@unitn.it Abstract In this paper we present a novel application of compositional distributional semantic models (CDSMs): prediction of lexical typology. The paper introduces the notion of typological closeness, which is a novel rigorous formalization of semantic similarity based on com- parison of multilingual data. Starting from the Moscow Database of Qualitative Features for adjective typology, we create four datasets of typological closeness, on which we test a range of distributional semantic models. We show that, on the one hand, vector representa- tions of phrases based on data from one language can be used to predict how words within the phrase translate into different languages, and, on the other hand, that typological data can serve as a semantic benchmark for distributional models. We find that compositional distributional models, especially parametric ones, perform way above non-compositional alternatives on the task. Keywords: compositional distributional models, adjectives, lexical typology 1. Introduction It is well-known that meanings of words can vary from lan- guage to language. For example, there is no Russian ad- jective that could be used to translate the English adjective sharp in all contexts: sharp edge, sharp needle, sharp turn, sharp image, sharp mind, etc., and similarly no English adjective translates the Russian ostryj ‘sharp’ in all con- texts, which include among others ostryj rog ‘sharp horn’ and ostryj perec ‘hot pepper’. Can independently motivated semantic models predict such variation of lexical seman- tics across languages? For example, do existing semantic models provide clues suggesting that the adjective sharp in many but not all languages expresses the meaning of ‘spicy’? This question has no obvious answer: semantic models are usually built for one particular language and as such are not meant to account for linguistic variation. How- ever, it has been argued that patterns like ‘sharp’→‘spicy’ are not arbitrary but are motivated by an inherent semantic basis, which is common for different languages (Rakhilina and Reznikova, 2013; Koptjevskaja-Tamm et al., 2015). If linguists are right about this, an adequate model of natural language semantics should be able to predict such patterns. In search for the answer to our question, we evaluate sev- eral (compositional) distributional semantic models against data on lexical semantic typology of adjectives. 1.1. Distributional Semantic Models Distributional semantic models (DSMs) are able to pre- dict many kinds of phenomena, including word similarity, lexical ambiguity, and semantic priming, or even to pass TOEFL synonymy and analogy tests (Landauer and Du- mais, 1997; Griffiths et al., 2007; Turney and Pantel, 2010). Compositional distributional semantic models (CDSMs) at- tempt to extend vector representations to arbitrary phrases (Mitchell and Lapata, 2008; Baroni and Zamparelli, 2010), up to full sentences (Socher et al., 2011; Socher et al., 2012; Pham et al., 2013; Paperno …

Content truncated to 500 words