Biased TextRank: Unsupervised Graph-Based Content Extraction

Proceedings of the 28th International Conference on Computational Linguistics, pages 1642–1652 Barcelona, Spain (Online), December 8-13, 2020 1642 Biased TextRank: Unsupervised Graph-Based Content Extraction Ashkan Kazemi, Ver´onica P´erez-Rosas, Rada Mihalcea Department of Computer Science & Engineering University of Michigan, Ann Arbor {ashkank, vrncapr, mihalcea}@umich.edu Abstract We introduce Biased TextRank, a graph-based content extraction method inspired by the popular TextRank algorithm that ranks text spans according to their importance for language processing tasks and according to their relevance to an input “focus.” Biased TextRank enables focused content extraction for text by modifying the random restarts in the execution of TextRank. The random restart probabilities are assigned based on the relevance of the graph nodes to the focus of the task. We present two applications of Biased TextRank: focused summarization and expla- nation extraction, and show that our algorithm leads to improved performance on two different datasets by significant ROUGE-N score margins. Much like its predecessor, Biased TextRank is unsupervised, easy to implement and orders of magnitude faster and lighter than current state-of- the-art Natural Language Processing methods for similar tasks. 1 Introduction Content and information extraction are central to many Natural Language Processing (NLP) tasks, from question answering (Rajpurkar et al., 2018; Reddy et al., 2019) to text summarization (Hermann et al., 2015; Dang, 2005) and beyond. While the state-of-the-art solutions for these tasks mainly rely on training neural network architectures on very large datasets, there have been questions around the sustainability of these solutions and their effects on the environment. As highlighted in work by Strubell et al. (2019), training one large transformer-based model produces approximately four times more CO2 emissions than a car in its lifetime. These considerable negative environmental outcomes call for lighter and less resource-intensive alternative methods. TextRank (Mihalcea and Tarau, 2004) is a light-weight unsupervised graph-based content extraction algorithm that was initially designed for summarization and keyword extraction applications. Since its introduction, it has been adapted and used in numerous other applications and settings, including opinion mining (Petasis and Karkaletsis, 2016; Deguchi and Yamaguchi, 2019), credibility assessment (Balcerzak et al., 2014) and lyrics summarization (Son and Shin, 2018), among others. Most recently, TextRank has been included in the latest release of the popular spaCy library.1 There have been online tutorials and updating studies (Barrios et al., 2015) that demonstrate TextRank’s relevance years after its initial release. Some of the TextRank extensions that have been proposed in recent years rely on the idea of per- sonalized (or topic-sensitive) PageRank (Haveliwala, 2003) and its successor algorithms. For instance, PositionRank (Florescu and Caragea, 2017) changed the TextRank rankings to account for the position of candidate words in the input document, and showed that this position-aware algorithm led to improve- ments in keyword extraction over TextRank and over several other baselines. In this paper, we introduce Biased TextRank, which relies on document representation models and similarity measures that enable capturing meaning closeness between graph nodes and a target (focus) This work is licensed under a Creative Commons Attribution 4.0 International …

Content truncated to 500 words