Capturing Anomalies in the Choice of Content Words in Compositional Distributional Semantic Space

Proceedings of Recent Advances in Natural Language Processing, pages 365–372, Hissar, Bulgaria, 7-13 September 2013. Capturing Anomalies in the Choice of Content Words in Compositional Distributional Semantic Space Ekaterina Kochmar Computer Laboratory University of Cambridge ek358@cl.cam.ac.uk Ted Briscoe Computer Laboratory University of Cambridge ejb@cl.cam.ac.uk Abstract In this work, we present a new task for testing compositional distributional se- mantic models. Recently, there has been a spate of research into how distributional representations of individual words can be combined to represent the meaning of phrases. Vecchi et al. (2011) have shown that some compositional models, includ- ing the additive and multiplicative mod- els of Mitchell and Lapata (2008; 2010) and the linear map-based model of Baroni and Zamparelli (2010), can be applied to detect semantically anomalous adjective– noun combinations. We extend their ex- periments and apply these models to the combinations extracted from texts written by learners of English. Our work contributes to the field of com- positional distributional semantics by in- troducing a new test paradigm for seman- tic models and shows how these models can be used for error detection in language learners’ content word combinations. 1 Introduction Vector-based (distributional) models are widely used for representing the meaning of single words. They rely on the assumption that word meaning can be learned from the linguistic environment and can be approximated by a word’s distribution across contexts. Words are represented as vectors in a high-dimensional space, with vector dimen- sions encoding word co-occurrence with contex- tual elements – other words within a local win- dow, words linked by specific dependencies to the target word, and so forth. Distributional models provide a clear basis for interpreting word mean- ing, as well as a simple means for measuring se- mantic similarity. These properties have been ex- ploited in many NLP tasks, including automatic thesaurus extraction (Grefenstette, 1994), word sense induction (Sch¨utze, 1998) and disambigua- tion (McCarthy et al., 2004), collocation extrac- tion (Schone and Jurafsky, 2001) and others. In contrast to single words, the distribution of phrases cannot be used as a reliable approxima- tion of their meaning, as phrase vectors are much sparser. Irrespective of the size of the corpus con- sidered, some content word combinations will re- main unattested as a consequence of their Zipf-like distributions. For example, Vecchi et al. (2011) have shown that both semantically acceptable and semantically deviant word combinations will be absent from large English corpora. A promising alternative is to use compositional models which combine distributional vectors for the component words in some way, for example, using a di- rect vector combination function (Kintsch, 2001; Mitchell and Lapata, 2008; Erk and Pad´o, 2008; Thater et al., 2010) or linear transformations on vectors (Baroni and Zamparelli, 2010). In spite of the spate of recent work in this area, the question of how to combine word representa- tions is far from answered. Compositional mod- els can be assessed by their ability both to provide a solid theoretical basis for meaning composition and to represent composite meaning for …

Content truncated to 500 words