Back to the Basics: A Quantitative Analysis of Statistical and Graph-Based Term Weighting Schemes for Keyword Extraction

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8089–8103 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 8089 Back to the Basics: A Quantitative Analysis of Statistical and Graph-Based Term Weighting Schemes for Keyword Extraction Asahi Ushio and Federico Liberatore and Jose Camacho-Collados School of Computer Science and Informatics Cardiff University, United Kingdom {UshioA,LiberatoreF,CamachoColladosJ}@cardiff.ac.uk Abstract Term weighting schemes are widely used in Natural Language Processing and Information Retrieval. In particular, term weighting is the basis for keyword extraction. However, there are relatively few evaluation studies that shed light about the strengths and shortcomings of each weighting scheme. In fact, in most cases researchers and practitioners resort to the well- known tf-idf as default, despite the existence of other suitable alternatives, including graph- based models. In this paper, we perform an exhaustive and large-scale empirical compar- ison of both statistical and graph-based term weighting methods in the context of keyword extraction. Our analysis reveals some interest- ing findings such as the advantages of the less- known lexical specificity with respect to tf-idf, or the qualitative differences between statisti- cal and graph-based methods. Finally, based on our findings we discuss and devise some suggestions for practitioners.1 1 Introduction Keyword extraction has been an essential task in many scientific fields as a first step to extract rele- vant terms from text corpora. Despite the simplicity of the task, it still poses practical problems, and often researchers resort to simple but reliable tech- niques such as tf-idf (Jones, 1972). In turn, term weighting schemes such as tf-idf paved the way for developing large-scale Information Retrieval (IR) systems (Ramos et al., 2003; Wu et al., 2008). Its simple formulation is still widely used nowadays, not only for keyword extraction but also as an im- portant component in IR (Jabri et al., 2018; Marcos- Pablos and García-Peñalvo, 2020) and Natural Lan- guage Processing (NLP) tasks (Riedel et al., 2017; Arroyo-Fernández et al., 2019). 1Source code to reproduce our experimental results, includ- ing a keyword extraction library, are available in the following repository: https://github.com/asahi417/kex While there exist supervised and neural tech- niques (Lahiri et al., 2017; Xiong et al., 2019; Sun et al., 2020), as well as ensembles of unsupervised methods (Campos et al., 2020; Tang et al., 2020) that can provide competitive performance, in this paper we go back to the basics and analyze in detail the single components of unsupervised methods for keyword extraction. In fact, it is still common to rely on unsupervised methods for keyword extrac- tion given their versatility and the lack of training sets in specialized domains. In order to fill this gap, in this paper we per- form an extensive analysis of single unsupervised keyword extraction techniques in a wide range of settings and datasets. To the best of our knowledge, this is the first large-scale empirical evaluation per- formed across base statistical and graphical key- word extraction methods. Our analysis sheds light on some properties of statistical methods largely unknown. For instance, our experiments …

Content truncated to 500 words