Sentence Centrality Revisited for Unsupervised Summarization

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6236–6247 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 6236 Sentence Centrality Revisited for Unsupervised Summarization Hao Zheng and Mirella Lapata Institute for Language, Cognition and Computation School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB Hao.Zheng@ed.ac.uk mlap@inf.ed.ac.uk Abstract Single document summarization has enjoyed renewed interest in recent years thanks to the popularity of neural network models and the availability of large-scale datasets. In this pa- per we develop an unsupervised approach ar- guing that it is unrealistic to expect large-scale and high-quality training data to be available or created for different types of summaries, domains, or languages. We revisit a popu- lar graph-based ranking algorithm and mod- ify how node (aka sentence) centrality is com- puted in two ways: (a) we employ BERT, a state-of-the-art neural representation learning model to better capture sentential meaning and (b) we build graphs with directed edges argu- ing that the contribution of any two nodes to their respective centrality is influenced by their relative position in a document. Experimental results on three news summarization datasets representative of different languages and writ- ing styles show that our approach outperforms strong baselines by a wide margin.1 1 Introduction Single-document summarization is the task of generating a shorter version of a document while retaining its most important content (Nenkova et al., 2011). Modern neural network-based ap- proaches (Nallapati et al., 2016; Paulus et al., 2018; Nallapati et al., 2017; Cheng and Lapata, 2016; See et al., 2017; Narayan et al., 2018b; Gehrmann et al., 2018) have achieved promis- ing results thanks to the availability of large- scale datasets containing hundreds of thousands of document-summary pairs (Sandhaus, 2008; Her- mann et al., 2015b; Grusky et al., 2018). Neverthe- less, it is unrealistic to expect that large-scale and high-quality training data will be available or cre- 1Our code is available at https://github.com/ mswellhao/PacSum. ated for different summarization styles (e.g., high- lights vs. single-sentence summaries), domains (e.g., user- vs. professionally-written articles), and languages. It therefore comes as no surprise that unsuper- vised approaches have been the subject of much previous research (Marcu, 1997; Radev et al., 2000; Lin and Hovy, 2002; Mihalcea and Tarau, 2004; Erkan and Radev, 2004; Wan, 2008; Wan and Yang, 2008; Hirao et al., 2013; Parveen et al., 2015; Yin and Pei, 2015; Li et al., 2017). A very popular algorithm for extractive single-document summarization is TextRank (Mihalcea and Tarau, 2004); it represents document sentences as nodes in a graph with undirected edges whose weights are computed based on sentence similarity. In or- der to decide which sentence to include in the sum- mary, a node’s centrality is often measured using graph-based ranking algorithms such as PageRank (Brin and Page, 1998). In this paper, we argue that the centrality mea- sure can be improved in two important respects. Firstly, to better capture sentential meaning and compute sentence similarity, we employ BERT (Devlin et al., …

Content truncated to 500 words