Perceptually Grounded Selectional Preferences

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 950–960, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics Perceptually grounded selectional preferences Ekaterina Shutova Computer Laboratory University of Cambridge, UK es407@cam.ac.uk Niket Tandon Max Planck Institute for Informatics, Germany ntandon@mpi-inf.mpg.de Gerard de Melo IIIS Tsinghua University, China gdm@demelo.org Abstract Selectional preferences (SPs) are widely used in NLP as a rich source of semantic information. While SPs have been tradi- tionally induced from textual data, human lexical acquisition is known to rely on both linguistic and perceptual experience. We present the first SP learning method that si- multaneously draws knowledge from text, images and videos, using image and video descriptions to obtain visual features. Our results show that it outperforms linguistic and visual models in isolation, as well as the existing SP induction approaches. 1 Introduction Selectional preferences (SPs) are the semantic con- straints that a predicate places onto its arguments. This means that certain classes of entities are more likely to fill the predicate’s argument slot than oth- ers. For instance, while the sentences “The au- thors wrote a new paper.” and “The cat is eating your sausage!” sound natural and describe plausi- ble real-life situations, the sentences “The carrot ate the keys.” and “The law sang a driveway.” ap- pear implausible and difficult to interpret, as the arguments do not satisfy the verbs’ common pref- erences. SPs provide generalisations about word meaning and use and find a wide range of appli- cations in natural language processing (NLP), in- cluding word sense disambiguation (Resnik, 1997; McCarthy and Carroll, 2003; Wagner et al., 2009), resolving ambiguous syntactic attachments (Hindle and Rooth, 1993), semantic role labelling (Gildea and Jurafsky, 2002; Zapirain et al., 2010), natural language inference (Zanzotto et al., 2006; Pantel et al., 2007), and figurative language processing (Fass, 1991; Mason, 2004; Shutova et al., 2013; Li et al., 2013). Automatic acquisition of SPs from linguistic data has thus become an active area of research. The community has investigated a range of techniques to tackle data sparsity and to per- form generalisation from observed arguments to their underlying types, including the use of Word- Net synsets as SP classes (Resnik, 1993; Li and Abe, 1998; Clark and Weir, 1999; Abney and Light, 1999; Ciaramita and Johnson, 2000), word cluster- ing (Rooth et al., 1999; Bergsma et al., 2008; Sun and Korhonen, 2009), distributional similarity met- rics (Erk, 2007; Peirsman and Pad´o, 2010), latent variable models ( ´O S´eaghdha, 2010; Ritter et al., 2010), and neural networks (Van de Cruys, 2014). Little research, however, has been concerned with the sources of knowledge that underlie the learning of SPs. There is ample evidence in cogni- tive and neurolinguistics that our concept learning and semantic representation are grounded in per- ception and action (Barsalou, 1999; Glenberg and Kaschak, 2002; Barsalou, 2008; Aziz-Zadeh and Damasio, 2008). This suggests that word mean- ing and relational knowledge are acquired not only …

Content truncated to 500 words