Improving Unsupervised Extractive Summarization with Facet-Aware Modeling

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1685–1697 August 1–6, 2021. ©2021 Association for Computational Linguistics 1685 Improving Unsupervised Extractive Summarization with Facet-Aware Modeling Xinnian Liang1∗, Shuangzhi Wu2, Mu Li2 and Zhoujun Li1† 1State Key Lab of Software Development Environment, Beihang University, Beijing, China 2Tencent Cloud Xiaowei, Beijing, China {xnliang,lizj}@buaa.edu.cn; {frostwu,ethanlli}@tencent.com; Abstract Unsupervised extractive summarization aims to extract salient sentences from documents without labeled corpus. Existing methods are mostly graph-based by computing sentence centrality. These methods usually tend to select sentences within the same facet, how- ever, which often leads to the facet bias prob- lem especially when the document has mul- tiple facets (i.e. long-document and multi- documents). To address this problem, we proposed a novel facet-aware centrality-based ranking model. We let the model pay more attention to different facets by introducing a sentence-document weight. The weight is added to the sentence centrality score. We evaluate our method on a wide range of sum- marization tasks that include 8 representa- tive benchmark datasets. Experimental re- sults show that our method consistently out- performs strong baselines especially in long- and multi-document scenarios and even per- forms comparably to some supervised mod- els. Extensive analyses confirm that the perfor- mance gains come from alleviating the facet bias problem. 1 Introduction Document summarization is the task of transform- ing a long document into a shorter version while retaining its most important content (Nenkova and McKeown, 2011).Existing extractive or abstractive methods are mostly in supervised fashion which rely on large amounts of labeled corpora (Cheng and Lapata, 2016; Nallapati et al., 2017; Gehrmann et al., 2018; Liu and Lapata, 2019a,b; Zhang et al., 2019; Wang et al., 2020). However, this is not available for different summarization styles, do- mains, and languages. Fortunately, recent work has shown successful practices on unsupervised *Contribution during internship at Tencent Inc. †Corresponding Author Figure 1: Examples from New York Times. We se- lected part of key sentences from the source document to show in this table. “...” refers to the omissions of context sentences due to space limitation. extractive summarization (Radev et al., 2000; Mi- halcea and Tarau, 2004; Erkan and Radev, 2004; Schluter and Søgaard, 2015; Tixier et al., 2017; Zheng and Lapata, 2019; Xu et al., 2020; Dong et al., 2020). Compare with supervised ones, unsu- pervised methods 1). remove the dependency on large-scale annotated document-summary pairs; 2). are more general for various scenarios. Graph-based models are commonly used in un- supervised extractive methods (Radev et al., 2000; Mihalcea and Tarau, 2004; Erkan and Radev, 2004). For example, Zheng and Lapata (2019) proposed a directed centrality-based method named PacSum 1686 Figure 2: Visualization of facet bias. Nodes refer to sentence representations and star is the document repre- sentation. Black solid circles mean facets. Red dashed circle means threshold in Section 3.1. The dashed bi- direction arrows denote the sentence similarities. by assuming that the contribution of any two nodes to their respective centrality is influenced by their relative position in a document. Dong …

Content truncated to 500 words