Visual Objects As Context: Exploiting Visual Objects for Lexical Entailment

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2723–2735 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 2723 Visual Objects As Context: Exploiting Visual Objects for Lexical Entailment Masayasu Muraoka mmuraoka@jp.ibm.com Tetsuya Nasukawa nasukawa@jp.ibm.com IBM Research Bishwaranjan Bhattacharjee bhatta@us.ibm.com Abstract We propose a new word representation method derived from visual objects in associated im- ages to tackle the lexical entailment task. Although it has been shown that the Dis- tributional Informativeness Hypothesis (DIH) holds on text, in which the DIH assumes that a context surrounding a hyponym is more in- formative than that of a hypernym, it has never been tested on visual objects. Since our per- ception is tightly associated with language, it is meaningful to explore whether the DIH holds on visual objects. To this end, we con- sider visual objects as the context of a word and represent a word as a bag of visual ob- jects found in images associated with the word. This allows us to test the feasibility of the visual DIH. To better distinguish word pairs in a hypernym relation from other relations such as co-hypernyms, we also propose a new measurable function that takes into account both the difference in the generality of mean- ing and similarity of meaning between words. Our experimental results show that the DIH holds on visual objects and that the proposed method combined with the proposed function outperforms existing unsupervised representa- tion methods. 1 Introduction Recognizing lexical entailment (LE) is a funda- mental component in Natural Language Processing (NLP), helping with many tasks such as textual entailment recognition (Garrette et al., 2011; Da- gan et al., 2013), taxonomy creation (Snow et al., 2006; Navigli et al., 2011), and metaphor detection (Mohler et al., 2013). Lexical entailment defines an asymmetric relation between two terms, where one term can be inferred by the other, but not vice versa. For example, dog entails animal but not vice versa because animal does not always mean dog. To recognize LE, it is required (1) to construct a good representation that captures the generality of the meaning of a term, and (2) to define a mea- sure to jointly calculate the asymmetric difference in the generality of meaning and the similarity of meaning between two given terms. An increasing number of representation methods and measures to compute hypernymy have been proposed to date (Weeds and Weir, 2003; Clarke, 2009; Kotlerman et al., 2009; Lenci and Benotto, 2012). Especially, Santus et al. (2014) and Rimell (2014) proposed unsupervised methods that fol- low the Distributional Informativeness Hypothesis (DIH). However, they have not used visual informa- tion in their methods and instead, required a large amount of textual data to construct the representa- tions. In the field of computer vision, Deselaers and Ferrari (2011) have shown that terms at higher levels in the hierarchy of WordNet (Miller, 1995) tend to correspond to a greater variety of images than terms at lower levels. Kiela et al. (2015a) have focused on this tendency and …

Content truncated to 500 words