Neural Extractive Summarization with Hierarchical Attentive Heterogeneous Graph Network

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 3622–3631, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 3622 Neural Extractive Summarization with Hierarchical Attentive Heterogeneous Graph Network Ruipeng Jia1,2, Yanan Cao1∗, Hengzhu Tang1, Fang Fang1∗, Cong Cao1 and Shi Wang3 1Institute of Information Engineering, Chinese Academy of Sciences 2School of Cyber Security, University of Chinese Academy of Sciences 3Institute of Computing Technology, Chinese Academy of Sciences 1,2{jiaruipeng, caoyanan, tanghengzhu, fangfang0703, caocong}@iie.ac.cn 3wangshi@ict.ac.cn Abstract Sentence-level extractive text summarization is substantially a node classification task of network mining, adhering to the infor- mative components and concise representa- tions. There are lots of redundant phrases between extracted sentences, but it is diffi- cult to model them exactly by the general supervised methods. Previous sentence en- coders, especially BERT, specialize in mod- eling the relationship between source sen- tences. While, they have no ability to con- sider the overlaps of the target selected sum- mary, and there are inherent dependencies among target labels of sentences. In this pa- per, we propose HAHSum (as shorthand for Hierarchical Attentive Heterogeneous Graph for Text Summarization), which well mod- els different levels of information, includ- ing words and sentences, and spotlights re- dundancy dependencies between sentences. Our approach iteratively refines the sen- tence representations with redundancy-aware graph and delivers the label dependencies by message passing. Experiments on large scale benchmark corpus (CNN/DM, NYT, and NEWSROOM) demonstrate that HAHSum yields ground-breaking performance and out- performs previous extractive summarizers. 1 Introduction Single document extractive summarization aims to select subset sentences and assemble them as infor- mative and concise summaries. Recent advances (Nallapati et al., 2017; Zhou et al., 2018; Liu and Lapata, 2019; Zhong et al., 2020) focus on balanc- ing the salience and redundancy of sentences, i.e. selecting the sentences with high semantic similar- ity to the gold summary and resolving the redun- dancy between selected sentences. Taking Table ∗Corresponding authors: Yanan Cao and Fang Fang Salience Label Sentence sent1: 0.7 0 Deanna Holleran is charged in murder. sent2: 0.1 0 Jackson County Prosecutor Jean Peters Baker announced today. sent3: 0.7 1 Deanna Holleran faces a charge of traffic accident. sent4: 0.7 1 The fatal traffic accident is a murder. sent5: 0.2 0 It took the life of Marianna Hernandez near 9th Hardesty. Summary: Woman faces a charge of murder for a fatal traffic accident. Table 1: Simplified News from Jackson County Prose- cutor. Salience score is an approximate estimation de- rived from semantic and Label is converted from gold summary to ensure the concision and accuracy of the extracted summaries. 1 for example, there are five sentences in a docu- ment, and each of them is assigned one salience score and one label indicating whether this sentence should be contained in the extracted summary. Al- though sent1, sent3, and sent4 are assigned high salience score, just sent3 and sent4 are selected as the summary sentences (with label 1) because there are too much redundancy information between un- selected sent1 and selected sent3. That …

Content truncated to 500 words