Discourse-Aware Neural Extractive Text Summarization

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5021–5031 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 5021 Discourse-Aware Neural Extractive Text Summarization Jiacheng Xu∗1, Zhe Gan2, Yu Cheng2, Jingjing Liu2 1The University of Texas at Austin 2Microsoft Dynamics 365 AI Research jcxu@cs.utexas.edu; {zhe.gan,yu.cheng,jingjl}@microsoft.com Abstract Recently BERT has been adopted for doc- ument encoding in state-of-the-art text sum- marization models. However, sentence-based extractive models often result in redundant or uninformative phrases in the extracted summaries. Also, long-range dependencies throughout a document are not well cap- tured by BERT, which is pre-trained on sen- tence pairs instead of documents. To address these issues, we present a discourse-aware neural summarization model - DISCOBERT1. DISCOBERT extracts sub-sentential discourse units (instead of sentences) as candidates for extractive selection on a finer granularity. To capture the long-range dependencies among discourse units, structural discourse graphs are constructed based on RST trees and corefer- ence mentions, encoded with Graph Convolu- tional Networks. Experiments show that the proposed model outperforms state-of-the-art methods by a significant margin on popular summarization benchmarks compared to other BERT-base models. 1 Introduction Neural networks have achieved great success in the task of text summarization (Nenkova et al., 2011; Yao et al., 2017). There are two main lines of research: abstractive and extractive. While the abstractive paradigm (Rush et al., 2015; See et al., 2017; Celikyilmaz et al., 2018; Sharma et al., 2019) focuses on generating a summary word-by-word after encoding the full document, the extractive approach (Cheng and Lapata, 2016; Zhou et al., 2018; Narayan et al., 2018) directly selects sen- tences from the document to assemble into a sum- mary. The abstractive approach is more flexible ∗ Most of this work was done when the first author was an intern at Microsoft. 1Code, illustration and datasets are available at: https://github.com/jiacheng-xu/DiscoBERT. 1. [It is one of the most prestigious honors]1 [bestowed upon journalists and people in the arts.]2 2. [And today, the Pulitzer prize for journalism went to The Post and Courier newspaper of Charleston, South Carolina,]1 [which has a tiny staff of just 80 and a daily circulation of 85,000.]2 …… 5. [Winner: ]1 [This iconic photo by New York Times photographer Daniel Berehulak, was part of a winning series,]2 [and shows James Dorbor, 8,]3 [suspected of being infected with Ebola,]4 [being carried by medical staff to an Ebola treatment center in Monrovia, Liberia.]5 …… 20. [The Pulitzer prizes,]1 [awarded annually by Columbia University,]2 [recognize extraordinary work in U.S. journalism, literature, drama, and other categories.]3 …… 22. [Other winners of the coveted award included the St. Louis Post-Dispatch.]1 …… Coref Graph RST Graph Document … … … … … … … … [EDU Selection] Mentions of ‘Pulitzer prizes’ 5. [Winner: ]1 [This iconic photo by New York Times photographer Daniel Berehulak, was part of a winning series,]2 [and shows James Dorbor, 8,]3 [suspected of being infected with Ebola,]4 [being carried by medical staff to an Ebola treatment center in Monrovia, Liberia.]5 1. [It is one …

Content truncated to 500 words