Improving Abstractive Document Summarization with Salient Information Modeling

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2132–2141 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 2132 Improving Abstractive Document Summarization with Salient Information Modeling Yongjian You1,2, Weijia Jia2, 1*, Tianyi Liu1,2, Wenmian Yang1,2 1Department of Computer Science and Engineering, Shanghai Jiao Tong University 2State Key Lab of IoT for Smart City, CIS, University of Macau, Macao, SAR China {youyongjian, jiawj, liutianyi, sdq11111}@sjtu.edu.cn Abstract Comprehensive document encoding and salient information selection are two major difficulties for generating summaries with adequate salient information. To tackle the above difficulties, we propose a Transformer- based encoder-decoder framework with two novel extensions for abstractive document summarization. Specifically, (1) to encode the documents comprehensively, we design a focus-attention mechanism and incorporate it into the encoder. This mechanism models a Gaussian focal bias on attention scores to enhance the perception of local context, which contributes to producing salient and informative summaries. (2) To distinguish salient information precisely, we design an independent saliency-selection network which manages the information flow from encoder to decoder. This network effectively reduces the influences of secondary information on the generated summaries. Experimental results on the popular CNN/Daily Mail benchmark demonstrate that our model outperforms other state-of-the-art baselines on the ROUGE metrics. 1 Introduction Document summarization is a fundamental task of natural language generation which con- denses the given documents and generates flu- ent summaries with salient information automat- ically. Recent successes of neural sequence-to- sequence (seq2seq) models (Luong et al., 2015; Wu et al., 2016; Tu et al., 2016) enable the end- to-end framework for natural language genera- tion, which inspires the research on abstractive summarization. Abstractive document summa- rization employs an end-to-end language model to encode a document into high-dimensional rep- resentations and then decode the representations into an abstractive summary. Though promis- Documents: a [duke student] has [admitted to hanging a noose made of rope] from a tree near a student union , [university of- ficials] said thursday . the prestigious private school did n’t identify the student , citing federal privacy laws . in a news release , it said the student was [no longer] on cam- pus and [will face] student conduct [review] . the [student was identified during an investigation] by campus police and the office of student affairs and admitted to placing the noose on the tree early wednesday , the university said . ... at a forum held on the steps of duke chapel , close to where [the noose was discovered at 2 a.m]. , hundreds of people gathered . “ you came here for the reason that you want to say with me , ‘ this is no duke we will accept . ... Reference summary: student is no longer on duke university campus and will face disciplinary review . school officials identified student during investigation .and the person admitted to hanging the noose , duke says the noose , made of rope , was discovered on campus about 2 a.m. Table …

Content truncated to 500 words