Unsupervised Keyphrase Extraction via Interpretable Neural Networks

Findings of the Association for Computational Linguistics: EACL 2023, pages 1107–1119 May 2-6, 2023 ©2023 Association for Computational Linguistics Unsupervised Keyphrase Extraction via Interpretable Neural Networks Rishabh Joshi♣ ∗ Vidhisha Balachandran♣ ∗ Emily Saldanha♢ Maria Glenski♢ Svitlana Volkova♢ Yulia Tsvetkov♠ ♣Language Technologies Institute, Carnegie Mellon University ♢Pacific Northwest National Laboratory ♠Paul G. Allen School of Computer Science & Engineering, University of Washington {rjoshi2,vbalacha}@cs.cmu.edu, {emily.saldanha,maria.glenski,svitlana.volkova}@pnnl.gov, yuliats@cs.washington.edu Abstract Keyphrase extraction aims at automatically ex- tracting a list of “important” phrases represent- ing the key concepts in a document. Prior ap- proaches for unsupervised keyphrase extrac- tion resorted to heuristic notions of phrase im- portance via embedding clustering or graph centrality, requiring extensive domain exper- tise. Our work presents a simple alternative ap- proach which defines keyphrases as document phrases that are salient for predicting the topic of the document. To this end, we propose IN- SPECT—an approach that uses self-explaining models for identifying influential keyphrases in a document by measuring the predictive impact of input phrases on the downstream task of the document topic classification. We show that this novel method not only alleviates the need for ad-hoc heuristics but also achieves state-of- the-art results in unsupervised keyphrase ex- traction in four datasets across two domains: scientific publications and news articles.1 1 Introduction Keyphrase extraction is crucial for processing and analysis of long documents in specialized (e.g., scientific, medical) domains (Mekala and Shang, 2020; Betti et al., 2020; Wang et al., 2019). The task is challenging, as the notion of phrase impor- tance is context- and domain-dependent. There- fore, developing domain-agnostic keyphrase anno- tation guidelines and curating representative hand- labeled datasets is not feasible. This motivates the need for generalizable unsupervised approaches to keyphrase extraction. Unsupervised keyphrase extraction methods have used heuristic notions of phrase importance (Mihalcea and Tarau, 2004; Shang et al., 2018; Campos et al., 2018). Popular proxies for phrase importance include phrase clustering based on sta- tistical features like word density (Florescu and ∗Equal Contribution 1Code: https://github.com/rishabhjoshi/inspect. Figure 1: A comprehensive set of keyphrases should highlight important phrases for all major topics in a document. INSPECT identifies such keyphrases using in- terpretable neural models by measuring how use phrases are for predicting the topic of a text. Caragea, 2017a; Campos et al., 2018) and struc- tural features like graph centrality (Bougouin et al., 2013; Ding and Luo, 2022) or more recently neural embedding clustering techniques (Bennani-Smires et al., 2018; Zhang et al., 2022; Ding and Luo, 2021; Sun et al., 2020). However, such methods do not generalize to new domains as they require ex- perts to carefully construct domain-specific heuris- tics (Mani et al., 2020). Historically, topic models (Blei et al., 2001; Blei and McAuliffe, 2007; Wallach, 2006) have relied on salient words and phrases in a document, which are similar to the notion of keyphrases, although to the best of our knowledge there is no prior work that identified keyphrases using topic models. In this work, we hypothesize that end-to-end neu- ral models for topic classification latently rely …

Content truncated to 500 words