Cognitively Motivated Distributional Representations of Meaning

Cognitively Motivated Distributional Representations of Meaning Elias Iosif1,2, Spiros Georgiladakis1, and Alexandros Potamianos1,2 1School of Electrical & Computer Engineering, National Technical University of Athens, Greece 2“Athena” Research and Innovation Center, Greece iosife@central.ntua.gr, sgeorgil@central.ntua.gr, potam@central.ntua.gr Abstract Although meaning is at the core of human cognition, state-of-the-art distributional semantic models (DSMs) are often agnostic to the findings in the area of semantic cognition. In this work, we present a novel type of DSMs motivated by the dual–processing cognitive perspective that is triggered by lexico–semantic activations in the short–term human memory. The proposed model is shown to perform better than state-of-the-art models for computing semantic similarity between words. The fusion of different types of DSMs is also investigated achieving results that are comparable or better than the state-of-the-art. The used corpora along with a set of tools, as well as large repositories of vectorial word representations are made publicly available for four languages (English, German, Italian, and Greek). Keywords: distributional semantic models, activation based cognitive models, semantic similarity, multilinguality 1. Introduction Distributional semantic models (DSMs) aim to represent the meaning of lexical entities by encoding linguistic features extracted from text corpora. Word-level repre- sentations are the basis for more complex tasks such as phrase– and sentence–level representation and similarity computation (Mitchell and Lapata, 2010; Agirre et al., 2012). Related applications include paraphrase detection (Androutsopoulos and Malakasiotis, 2010), affective text analysis (Malandrakis et al., 2013), etc. Adopting the standpoint of (Potamianos, 2014) we ar- gue that the aforementioned tasks should be driven by the fundamental properties of the human cognitive sys- tem. Indeed, empirical evidence provided by cognitive sciences and psycholinguistics constitute a fertile source of inspiration for DSM design. In this spirit, we propose a novel type of word–level DSMs motivated by the dual– processing cognitive perspective (Kahneman, 2013) that is triggered by lexico–semantic activations in the short– term human memory. The proposed model is shown to perform better than state-of-the-art models for the prob- lem of word similarity computation with respect to vari- ous datasets in four languages (English, German, Italian, and Greek). The fusion of different types of DSMs is also investigated exceeding the performance of individ- ual models. We make publicly available the used corpora along with a set of tools, as well as large repositories of vectorial word representations for all four languages. 2. Related Work Word-level representations constitute the core aspect of DSMs typically constructed from co-occurrence statis- tics of word tuples. Word-level DSMs can be broadly categorized into unstructured and structured with respect to the extraction of contextual features. The bag-of- words model is the most widely used approach (e.g., see (Grefenstette, 1994)), lacking however some desirable characteristics such as “order sensitivity” (Turney, 2012). Unlike unstructured models, the order of extracted fea- tures is taken into account in the framework of struc- tured DSMs via the exploitation of syntactic relation- ships (e.g., argument structures and modifications) (Pado and Lapata, 2007). A comparison of unstructured and structured DSMs was conducted in (Agirre et al., 2009) for the task …

Content truncated to 500 words