Learning Abstract Concept Embeddings from Multi-Modal Data: Since You Probably Can’t See What I Mean

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 255–265, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Learning Abstract Concept Embeddings from Multi-Modal Data: Since You Probably Can’t See What I Mean Felix Hill Computer Laboratory University of Cambridge felix.hill@cl.cam.ac.uk Anna Korhonen Computer Laboratory University of Cambridge anna.korhonen@cl.cam.ac.uk Abstract Models that acquire semantic represen- tations from both linguistic and percep- tual input are of interest to researchers in NLP because of the obvious parallels with human language learning. Perfor- mance advantages of the multi-modal ap- proach over language-only models have been clearly established when models are required to learn concrete noun concepts. However, such concepts are comparatively rare in everyday language. In this work, we present a new means of extending the scope of multi-modal models to more commonly-occurring abstract lexical con- cepts via an approach that learns multi- modal embeddings. Our architecture out- performs previous approaches in combin- ing input from distinct modalities, and propagates perceptual information on con- crete concepts to abstract concepts more effectively than alternatives. We discuss the implications of our results both for op- timizing the performance of multi-modal models and for theories of abstract con- ceptual representation. 1 Introduction Multi-modal models that learn semantic represen- tations from both language and information about the perceptible properties of concepts were orig- inally motivated by parallels with human word learning (Andrews et al., 2009) and evidence that many concepts are grounded in perception (Barsa- lou and Wiemer-Hastings, 2005). The perceptual information in such models is generally mined di- rectly from images (Feng and Lapata, 2010; Bruni et al., 2012) or from data collected in psychologi- cal studies (Silberer and Lapata, 2012; Roller and Schulte im Walde, 2013). By exploiting the additional information en- coded in perceptual input, multi-modal models can outperform language-only models on a range of semantic NLP tasks, including modelling sim- ilarity (Bruni et al., 2014; Kiela et al., 2014) and free association (Silberer and Lapata, 2012), pre- dicting compositionality (Roller and Schulte im Walde, 2013) and concept categorization (Silberer and Lapata, 2014). However, to date, these pre- vious approaches to multi-modal concept learning focus on concrete words such as cat or dog, rather than abstract concepts, such as curiosity or loyalty. However, differences between abstract and con- crete processing and representation (Paivio, 1991; Hill et al., 2013; Kiela et al., 2014) suggest that conclusions about concrete concept learning may not necessarily hold in the general case. In this pa- per, we therefore focus on multi-modal models for learning both abstract and concrete concepts. Although concrete concepts might seem more basic or fundamental, the vast majority of open- class, meaning-bearing words in everyday lan- guage are in fact abstract. 72% of the noun or verb tokens in the British National Corpus (Leech et al., 1994) are rated by human judges1 as more abstract than the noun war, for instance, a con- cept many would already consider to be quite abstract. Moreover, abstract concepts by defi- nition encode …

Content truncated to 500 words