Distilling Hypernymy Relations from Language Models: On the Effectiveness of Zero-Shot Taxonomy Induction

Proceedings of the 11th Joint Conference on Lexical and Computational Semantics, pages 151 - 156 July 14-15, 2022 ©2022 Association for Computational Linguistics Distilling Hypernymy Relations from Language Models: On the Effectiveness of Zero-Shot Taxonomy Induction Devansh Jain♠∗ Luis Espinosa Anke♢ ♠ Department of Computer Science and Information Systems, BITS Pilani, India ♢ CardiffNLP (Cardiff University) - AMPLYFI f20180798@pilani.bits-pilani.ac.in espinosa-ankel@cardiff.ac.uk Abstract In this paper, we analyze zero-shot taxonomy learning methods which are based on distilling knowledge from language models via prompt- ing and sentence scoring. We show that, de- spite their simplicity, these methods outperform some supervised strategies and are competitive with the current state-of-the-art under adequate conditions. We also show that statistical and linguistic properties of prompts dictate down- stream performance1. 1 Introduction Taxonomy learning (TL) is the task of arranging domain terminologies into hierarchical structures where terms are nodes and edges denote is-a (hyper- nymic) relationships (Hwang et al., 2012). Domain- specific concept generalization is at the core of hu- man cognition (Yu et al., 2015), and a key enabler in NLP tasks where inference and reasoning are important, e.g.: semantic similarity (Pilehvar et al., 2013; Yu and Dredze, 2014), WSD (Agirre et al., 2014) and, more recently, QA (Joshi et al., 2020) and NLI (Chen et al., 2020). Earlier approaches to taxonomy learning focused on mining lexico-syntactic patterns from candidate (hyponym, hypernym) pairs (Hearst, 1992; Snow et al., 2004; Kozareva and Hovy, 2010; Boella and Di Caro, 2013; Espinosa-Anke et al., 2016), cluster- ing (Yang and Callan, 2009), graph-based methods (Fountain and Lapata, 2012; Velardi et al., 2013) or word embeddings (Fu et al., 2014; Yu et al., 2015). These methods, which largely rely on hand-crafted features, are still relevant today, and complement modern approaches exploiting language models (LMs), either via sequence classification (Chen et al., 2021), or combining contextual, distributed, and lexico-syntactic features (Yu et al., 2020). In ∗ Work done during an internship at CardiffNLP. 1Code available at https://github.com/devanshrj/ zero-shot-taxonomy. parallel, several works have recently focused on us- ing LMs as zero-shot tools for solving NLP tasks, e.g., commonsense, relational and analogical rea- soning (Petroni et al., 2019; Bouraoui et al., 2020; Ushio et al., 2021; Paranjape et al., 2021), multi- word expression (MWE) identification (Espinosa- Anke et al., 2021; Garcia et al., 2021), QA (Shwartz et al., 2020; Banerjee and Baral, 2020), domain labeling (Sainz and Rigau, 2021), or lexical substi- tution and simplification (Zhou et al., 2019). More- over, by tuning and manipulating natural language queries (often referred to as prompts), impressive results have been recently obtained on tasks such as semantic textual similarity, entailment, or relation classification (Shin et al., 2020; Qin and Eisner, 2021). In this paper, we evaluate LMs on TL bench- marks using prompt-based and sentence-scoring techniques, and find not only that they are com- petitive with common approaches proposed in the literature (which are typically supervised and/or reliant on external resources), but that they achieve state-of-the-art results in certain domains. 2 Methodology We follow Ushio et al. (2021) …

Content truncated to 500 words