Learning Grounded Meaning Representations with Autoencoders

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 721–732, Baltimore, Maryland, USA, June 23-25 2014. c⃝2014 Association for Computational Linguistics Learning Grounded Meaning Representations with Autoencoders Carina Silberer and Mirella Lapata Institute for Language, Cognition and Computation School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB c.silberer@ed.ac.uk, mlap@inf.ed.ac.uk Abstract In this paper we address the problem of grounding distributional representations of lexical meaning. We introduce a new model which uses stacked autoencoders to learn higher-level embeddings from tex- tual and visual input. The two modali- ties are encoded as vectors of attributes and are obtained automatically from text and images, respectively. We evaluate our model on its ability to simulate similar- ity judgments and concept categorization. On both tasks, our approach outperforms baselines and related models. 1 Introduction Recent years have seen a surge of interest in sin- gle word vector spaces (Turney and Pantel, 2010; Collobert et al., 2011; Mikolov et al., 2013) and their successful use in many natural language ap- plications. Examples include information retrieval (Manning et al., 2008), search query expansions (Jones et al., 2006), document classification (Se- bastiani, 2002), and question answering (Yih et al., 2013). Vector spaces have been also popular in cognitive science figuring prominently in simula- tions of human behavior involving semantic prim- ing, deep dyslexia, text comprehension, synonym selection, and similarity judgments (see Griffiths et al., 2007). In general, these models specify mechanisms for constructing semantic representa- tions from text corpora based on the distributional hypothesis (Harris, 1970): words that appear in similar linguistic contexts are likely to have related meanings. Word meaning, however, is also tied to the physical world. Words are grounded in the exter- nal environment and relate to sensorimotor experi- ence (Regier, 1996; Landau et al., 1998; Barsalou, 2008). To account for this, new types of perceptu- ally grounded distributional models have emerged. These models learn the meaning of words based on textual and perceptual input. The latter is ap- proximated by feature norms elicited from humans (Andrews et al., 2009; Steyvers, 2010; Silberer and Lapata, 2012), visual information extracted automatically from images, (Feng and Lapata, 2010; Bruni et al., 2012a; Silberer et al., 2013) or a combination of both (Roller and Schulte im Walde, 2013). Despite differences in formulation, most existing models conceptualize the problem of meaning representation as one of learning from multiple views corresponding to different modali- ties. These models still represent words as vectors resulting from the combination of representations with different statistical properties that do not nec- essarily have a natural correspondence (e.g., text and images). In this work, we introduce a model, illus- trated in Figure 1, which learns grounded mean- ing representations by mapping words and im- ages into a common embedding space. Our model uses stacked autoencoders (Bengio et al., 2007) to induce semantic representations integrating vi- sual and textual information. The literature de- scribes several successful approaches to multi- modal learning using different variants of deep networks (Ngiam et al., …

Content truncated to 500 words