Supersense Embeddings: A Unified Model for Supersense Interpretation, Prediction, and Utilization

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 2029–2041, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Supersense Embeddings: A Unified Model for Supersense Interpretation, Prediction, and Utilization Lucie Flekova† and Iryna Gurevych†‡ † Ubiquitous Knowledge Processing Lab (UKP-TUDA) Department of Computer Science, Technische Universit¨at Darmstadt ‡ Ubiquitous Knowledge Processing Lab (UKP-DIPF) German Institute for Educational Research www.ukp.tu-darmstadt.de Abstract Coarse-grained semantic categories such as supersenses have proven useful for a range of downstream tasks such as question an- swering or machine translation. To date, no effort has been put into integrating the supersenses into distributional word rep- resentations. We present a novel joint em- bedding model of words and supersenses, providing insights into the relationship be- tween words and supersenses in the same vector space. Using these embeddings in a deep neural network model, we demon- strate that the supersense enrichment leads to a significant improvement in a range of downstream classification tasks. 1 Introduction The effort of understanding the meaning of words is central to the NLP community. The word sense dis- ambiguation (WSD) task has therefore received a substantial amount of attention (see Navigli (2009) or Pal and Saha (2015) for an overview). Words in training and evaluation data are usually annotated with senses taken from a particular lexical semantic resource, most commonly WordNet (Miller, 1995). However, WordNet has been criticized to provide too fine-grained distinctions for end level applica- tions. e.g. in machine translation or information retrieval (Izquierdo et al., 2009). Although some researchers report an improvement in sentiment pre- diction using WSD (Rentoumi et al., 2009; Akkaya et al., 2011; Sumanth and Inkpen, 2015), the pub- lication bias toward positive results (Plank et al., 2014) impedes the comparison to experiments with the opposite conclusion, and the contribution of WSD to downstream document classification tasks remains “mostly speculative”(Ciaramita and Altun, 2006), which can be attributed to the too subtle sense distinctions (Navigli, 2009). This is why su- persenses, the coarse-grained word labels based on WordNet’s (Fellbaum, 1998) lexicographer files, have recently gained attention for text classification tasks. Supersenses contain 26 labels for nouns, such as ANIMAL, PERSON or FEELING and 15 labels for verbs, such as COMMUNICATION, MOTION or COGNITION. Usage of supersense labels has been shown to improve dependency parsing (Agirre et al., 2011), named entity recognition (Marrero et al., 2009; R¨ud et al., 2011), non-factoid question answering (Surdeanu et al., 2011), question gen- eration (Heilman, 2011), semantic role labeling (Laparra and Rigau, 2013), personality profiling (Flekova and Gurevych, 2015), semantic similar- ity (Severyn et al., 2013) and metaphor detection (Tsvetkov et al., 2013). An alternative path to semantic interpretation follows the distributional hypothesis (Harris, 1954). Recently, word vector representations learned with neural-network based language models have con- tributed to state-of-the-art results on various lin- guistic tasks (Bordes et al., 2011; Mikolov et al., 2013b; Pennington et al., 2014; Levy et al., 2015). In this work, we present a novel approach for incorporating the supersense information into the word …

Content truncated to 500 words