Collaborative Ranking between Supervised and Unsupervised Approaches for Keyphrase Extraction

The 2014 Conference on Computational Linguistics and Speech Processing ROCLING 2014, pp. 110-124  The Association for Computational Linguistics and Chinese Language Processing Collaborative Ranking between Supervised and Unsupervised Approaches for Keyphrase Extraction Gerardo Figueroa, Yi-Shin Chen Abstract Automatic keyphrase extraction methods have generally taken either supervised or unsupervised approaches. Supervised methods extract keyphrases by using a training document set, thus acquiring knowledge from a global collection of texts. Conversely, unsupervised methods extract keyphrases by determining their relevance in a single-document context, without prior learning. We present a hybrid keyphrase extraction method for short articles, HybridRank, which leverages the benefits of both approaches. Our system implements modified versions of the TextRank (Mihalcea and Tarau, 2004)—unsupervised—and KEA (Witten et al., 1999)—supervised—methods, and applies a merging algorithm to produce an overall list of keyphrases. We have tested HybridRank on more than 900 abstracts belonging to a wide variety of subjects, and show its superior effectiveness. We conclude that knowledge collaboration between supervised and unsupervised methods can produce higher-quality keyphrases than applying these methods individually. Keywords: Keyword extraction, Keyphrase extraction, Hybrid approach, Supervised methods, Unsupervised methods 1. Introduction Keyphrases—also called keywords1—are highly condensed summaries that describe the contents of a document. They help readers know quickly what a document is about, and are generally assigned by the document's author or by a human indexer. However, with the massive growth of documents on the Web each day, it has become impractical to manually assign keywords to each document. The need for software applications that automatically assign keywords to documents has therefore become necessary.  Institute of Information Systems and Applications, National Tsing Hua University, Hsinchu, Taiwan E-mail: {gerardo.ofc, yishin}@gmail.com 1 A keyphrase is a phrase composed of one or more keywords. We will use the terms keyphrase and keyword interchangeably in this paper. 110 In this work we apply efficient and effective practices from supervised and unsupervised methods to produce a hybrid system HybridRank. On the supervised side, we implement an extension of the Naïve Bayes classifier originally proposed in KEA (Witten et al., 1999). This classifier has shown to be practical to implement and can be extended for improved effectiveness. On the unsupervised side, we apply the well-known TextRank (Mihalcea and Tarau, 2004) algorithm with some modifications. TextRank is similarly practical to implement, and can effectively extract keyphrases from texts regardless of their size or domain. Each method contributes by providing a list of keyphrases for a particular text, sorted by their rank or relevance as seen from each approach. Finally, a collaborative algorithm is executed, in which the two keyphrase lists are merged to create an overall list of keyphrases for that text. The merging algorithm thus takes into account the ranks given by both approaches to each keyphrase and produces a final, collaborative score reflected by these ranks. We have tested HybridRank on a large number of abstracts belonging to scientific papers across different domains. The results of our experiments show the effectiveness of the proposed method and of the improvements …

Content truncated to 500 words