Extractive Summarization as Text Matching

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6197–6208 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 6197 Extractive Summarization as Text Matching Ming Zhong∗, Pengfei Liu∗, Yiran Chen, Danqing Wang, Xipeng Qiu†, Xuanjing Huang Shanghai Key Laboratory of Intelligent Information Processing, Fudan University School of Computer Science, Fudan University 825 Zhangheng Road, Shanghai, China {mzhong18,pfliu14,yrchen19,dqwang18,xpqiu,xjhuang}@fudan.edu.cn Abstract This paper creates a paradigm shift with regard to the way we build neural extractive summa- rization systems. Instead of following the com- monly used framework of extracting sentences individually and modeling the relationship be- tween sentences, we formulate the extractive summarization task as a semantic text match- ing problem, in which a source document and candidate summaries will be (extracted from the original text) matched in a semantic space. Notably, this paradigm shift to seman- tic matching framework is well-grounded in our comprehensive analysis of the inherent gap between sentence-level and summary-level ex- tractors based on the property of the dataset. Besides, even instantiating the framework with a simple form of a matching model, we have driven the state-of-the-art extractive re- sult on CNN/DailyMail to a new level (44.41 in ROUGE-1). Experiments on the other five datasets also show the effectiveness of the matching framework. We believe the power of this matching-based summarization frame- work has not been fully exploited. To encour- age more instantiations in the future, we have released our codes, processed dataset, as well as generated summaries in https://github. com/maszhongming/MatchSum. 1 Introduction The task of automatic text summarization aims to compress a textual document to a shorter highlight while keeping salient information on the original text. In this paper, we focus on extractive summa- rization since it usually generates semantically and grammatically correct sentences (Dong et al., 2018; Nallapati et al., 2017) and computes faster. Currently, most of the neural extractive summa- rization systems score and extract sentences (or smaller semantic unit (Xu et al., 2019)) one by ∗These two authors contributed equally. †Corresponding author. Document Candidate Summary Gold Summary extract Semantic Space BERT BERT BERT Figure 1: MATCHSUM framework. We match the con- textual representations of the document with gold sum- mary and candidate summaries (extracted from the doc- ument). Intuitively, better candidate summaries should be semantically closer to the document, while the gold summary should be the closest. one from the original text, model the relationship between the sentences, and then select several sen- tences to form a summary. Cheng and Lapata (2016); Nallapati et al. (2017) formulate the ex- tractive summarization task as a sequence label- ing problem and solve it with an encoder-decoder framework. These models make independent bi- nary decisions for each sentence, resulting in high redundancy. A natural way to address the above problem is to introduce an auto-regressive decoder (Chen and Bansal, 2018; Jadhav and Rajan, 2018; Zhou et al., 2018), allowing the scoring operations of different sentences to influence on each other. Trigram Blocking (Paulus et al., 2017; Liu and La- pata, 2019), as …

Content truncated to 500 words