Exploiting Position and Contextual Word Embeddings for Keyphrase Extraction from Scientific Papers

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 1585–1591 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 1585 Exploiting Position and Contextual Word Embeddings for Keyphrase Extraction from Scientific Papers Krutarth Patel Computer Science Kansas State University Manhattan, Kansas, United States kipatel@ksu.edu Cornelia Caragea Computer Science University of Illinois at Chicago Chicago, Illinois, United States cornelia@uic.edu Abstract Keyphrases associated with research papers provide an effective way to find useful in- formation in the large and growing schol- arly digital collections. In this paper, we present KPRank, an unsupervised graph-based algorithm for keyphrase extraction that ex- ploits both positional information and contex- tual word embeddings into a biased PageRank. Our experimental results on five benchmark datasets show that KPRank that uses contex- tual word embeddings with additional position signal outperforms previous approaches and strong baselines for this task. 1 Introduction Keyphrase extraction is the task of automatically extracting a small set of descriptive words or phrases that can accurately summarize the topics discussed in a document (Hasan and Ng, 2010, 2014). Keyphrases are useful in many applica- tions such as document indexing and summariza- tion (Abu-Jbara and Radev, 2011; Qazvinian et al., 2010; Turney, 2003), topic tracking (Augenstein et al., 2017), contextual advertising (Yih et al., 2006), and opinion mining (Berend, 2011). Most of the previous approaches to keyphrase extraction are either supervised or unsupervised. While supervised approaches perform generally better (Kim et al., 2013), the unsupervised ones have the advantage that they do not require large human-annotated corpora for training reliable mod- els. Unsupervised keyphrase extraction methods usually use graph-based ranking algorithms such as PageRank that work on the word graph con- structed from the target document (Mihalcea and Tarau, 2004). Various PageRank extensions have been proposed that incorporate different types of information (Wan and Xiao, 2008; Gollapalli and Caragea, 2014). For example, Wan and Xiao (2008) proposed to incorporate a local neighborhood of the target document into the graph construction, with the neighborhood being determined based on the textual similarity between documents. Liu et al. (2010) exploited topical information to select keyphrases from all major topics. More recently, Mahata et al. (2018) proposed a theme-weighted biased PageRank, called Key2Vec, for keyphrase extraction. In Key2Vec, a theme-vector is com- puted by averaging the embeddings of words and phrases from the title of a scientific document to capture its theme and the PageRank is biased based on the similarity of candidate words or phrases to the computed theme vector. However, this model is oblivious to the position of words in a scientific document, in which more important words appear not only frequently, but also close to the beginning of the document (Florescu and Caragea, 2017). Inspired by the Transformer models (Vaswani et al., 2017) that infuse positional information into the word embeddings to produce embeddings with time signal, we propose an extension of Key2Vec that incorporates words’ positions into a biased PageRank. Moreover, different from Mahata et al. (2018), who used …

Content truncated to 500 words