comp-syn: Perceptually Grounded Word Embeddings with Color

Proceedings of the 28th International Conference on Computational Linguistics, pages 1744–1751 Barcelona, Spain (Online), December 8-13, 2020 1744 comp-syn: Perceptually Grounded Word Embeddings with Color Bhargav Srinivasa Desikan University of Chicago Knowledge Lab bhargav@uchicago.edu Tasker Hull Psiphon Inc, Toronto t.mackersy@psiphon.ca Ethan O. Nadler Stanford University KIPAC & Department of Physics enadler@stanford.edu Douglas Guilbeault University of California, Berkeley Haas Business School douglas.guilbeault@berkeley.edu Aabir Abubaker Kar University of Chicago Knowledge Lab aabir@uchicago.edu Mark Chu Columbia University mbc2165@columbia.edu Donald Ruggiero Lo Sardo Sony CSL Paris losardor@gmail.com Abstract Popular approaches to natural language processing create word embeddings based on textual co-occurrence patterns, but often ignore embodied, sensory aspects of language. Here, we intro- duce the Python package comp-syn, which provides grounded word embeddings based on the perceptually uniform color distributions of Google Image search results. We demonstrate that comp-syn significantly enriches models of distributional semantics. In particular, we show that (1) comp-syn predicts human judgments of word concreteness with greater accuracy and in a more interpretable fashion than word2vec using low-dimensional word–color embeddings, and (2) comp-syn performs comparably to word2vec on a metaphorical vs. literal word- pair classification task. comp-syn is open-source on PyPi and is compatible with mainstream machine-learning Python packages. Our package release includes word–color embeddings for over 40,000 English words, each associated with crowd-sourced word concreteness judgments. 1 Introduction The embodied cognition paradigm seeks to ground semantic processing in bodily, affective, and social experiences. This paradigm explains how sensory information contributes to semantic processing, either through metaphors involving references to sensory experience (Lakoff and Turner, 1989; Gallese and Lakoff, 2005) or through the simulation of sensory experience in mental imagery (Bergen, 2012). For example, color is pervasive in linguistic metaphors (e.g., “her bank accounts are in the red”), and primes a range of affective and interpretative responses (Mehta and Zhu, 2009; Elliot and Maier, 2014). Extant methods in computational linguistics are limited in their ability to advance the embodied cogni- tion paradigm due to their focus on text, which often precludes multi-modal analyses of images and other forms of sensory data—including color—involved in human meaning-making activities. Distributional semantics is a particularly prominent approach, wherein textual co-occurrence patterns are used to embed words in a high-dimensional space; the resulting “distance” between word embeddings correlates with semantic similarity (Landauer and Dumais, 1997). Although leveraging neural networks (Mikolov et al., This work is licensed under a Creative Commons Attribution 4.0 International Licence. Licence details here. 1745 Model Embedding based on Dimension Concreteness prediction Metaphor task comp-syn Perceptually uniform color distributions 8–16 Linear: R2 = 0.96 Nonlinear: ln L = 86 92% test set accuracy word2vec Textual co-occurrence ∼ 300 Linear: R2 = 0.17 Nonlinear: ln L = 76 95% test set accuracy Table 1: Comparison of comp-syn and word2vec and summary of our main results. 2013) and syntactic information (Levy and Goldberg, 2014) has led to recent progress, popular models like word2vec lack firm grounding in their use of sensory information. Moreover, high-dimensional word embeddings created by neural networks are difficult to interpret (S¸enel …

Content truncated to 500 words