Compositional Hyponymy with Positive Operators

Proceedings of Recent Advances in Natural Language Processing, pages 638–647, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_075 638 Compositional Hyponymy with Positive Operators Martha Lewis ILLC, University of Amsterdam m.a.f.lewis@uva.nl Abstract Language is used to describe concepts, and many of these concepts are hierarchical. Moreover, this hierarchy should be compati- ble with forming phrases and sentences. We use linear-algebraic methods that allow us to encode words as collections of vectors. The representations we use have an ordering, re- lated to subspace inclusion, which we interpret as modelling hierarchical information. The word representations built can be understood within a compositional distributional seman- tic framework, providing methods for compos- ing words to form phrase and sentence level representations. The resulting representations give competitive results on simple sentence- level entailment datasets. 1 Introduction Distributional semantics (Harris, 1954; Firth, 1957) is effective and important within the area of computational modelling of language, partic- ularly as regards to synonymy and paraphrasing. Within the field, at least two additional properties are desirable. Firstly, we would like a method by which we can compose vectors to form rep- resentations above the word level. Secondly, we would like a notion of lexical entailment, or hy- ponymy, with which we can capture a sense of the generality of concepts, and the notion of one concept being an instance of another. Further- more, we would like these two properties to inter- act nicely with one another, so that the hyponymy relation is not lost when words are composed. In Bankova et al. (2019) the authors provide the- ory describing a notion of hyponymy that inter- acts well with compositionality, but do not pro- vide experimental support. In Balkır et al. (2016) the authors suggest a measure of hyponymy based on entropy which also interacts well with compo- sitionality and provide experimental support. A compositional version of the distributional inclu- sion hypothesis (DIH) (Geffet and Dagan, 2005) is examined in Kartsaklis and Sadrzadeh (2016). In the current paper, we use the framework of Bankova et al. (2019) to build positive operators that represent words. The operators are built us- ing GloVe vectors (Pennington et al., 2014) and information about hyponym-hypernym relation- ships, which may be sourced either from human- curated resources like WordNet (Miller, 1995), or unsupervised sources, via the use of pattern-based methods. We give two new measures for graded hyponymy that provide a wider range of compar- isons than the entropy-derived measure developed in Balkır et al. (2016) or the eigenvalue-related measure of Bankova et al. (2019). We test our word representations and measures on a range of datasets. Three of the datasets are single-word entailment and have been de- signed to test directionality (BLESS) (Baroni and Lenci, 2011), detection (WBLESS) (Weeds et al., 2014), and both directionality and direction to- gether (BIBLESS) (Kiela et al., 2015). We also test our models on the compositional dataset of Kartsaklis and Sadrzadeh (2016). This dataset provides a test for entailment at the phrase and sentence level. We find that our model performs …

Content truncated to 500 words