Ranking Sentences for Extractive Summarization with Reinforcement Learning

Proceedings of NAACL-HLT 2018, pages 1747–1759 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Ranking Sentences for Extractive Summarization with Reinforcement Learning Shashi Narayan Shay B. Cohen Mirella Lapata Institute for Language, Cognition and Computation School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh, EH8 9AB shashi.narayan@ed.ac.uk, {scohen,mlap}@inf.ed.ac.uk Abstract Single document summarization is the task of producing a shorter version of a docu- ment while preserving its principal informa- tion content. In this paper we conceptualize extractive summarization as a sentence rank- ing task and propose a novel training algorithm which globally optimizes the ROUGE evalua- tion metric through a reinforcement learning objective. We use our algorithm to train a neu- ral summarization model on the CNN and Dai- lyMail datasets and demonstrate experimen- tally that it outperforms state-of-the-art extrac- tive and abstractive systems when evaluated automatically and by humans.1 1 Introduction Automatic summarization has enjoyed wide popu- larity in natural language processing due to its po- tential for various information access applications. Examples include tools which aid users navigate and digest web content (e.g., news, social media, product reviews), question answering, and person- alized recommendation engines. Single document summarization — the task of producing a shorter version of a document while preserving its infor- mation content — is perhaps the most basic of summarization tasks that have been identified over the years (see Nenkova and McKeown, 2011 for a comprehensive overview). Modern approaches to single document summa- rization are data-driven, taking advantage of the success of neural network architectures and their ability to learn continuous features without re- course to preprocessing tools or linguistic annota- tions. Abstractive summarization involves various text rewriting operations (e.g., substitution, dele- tion, reordering) and has been recently framed as a sequence-to-sequence problem (Sutskever et al., 2014). Central in most approaches (Rush et al., 2015; Chen et al., 2016; Nallapati et al., 2016; See 1Our code and data are available here: https://github. com/shashiongithub/Refresh. et al., 2017; Tan and Wan, 2017; Paulus et al., 2017) is an encoder-decoder architecture mod- eled by recurrent neural networks. The encoder reads the source sequence into a list of continuous- space representations from which the decoder gen- erates the target sequence. An attention mecha- nism (Bahdanau et al., 2015) is often used to lo- cate the region of focus during decoding. Extractive systems create a summary by identi- fying (and subsequently concatenating) the most important sentences in a document. A few re- cent approaches (Cheng and Lapata, 2016; Nalla- pati et al., 2017; Narayan et al., 2017; Yasunaga et al., 2017) conceptualize extractive summariza- tion as a sequence labeling task in which each label specifies whether each document sentence should be included in the summary. Existing mod- els rely on recurrent neural networks to derive a meaning representation of the document which is then used to label each sentence, taking the pre- viously labeled sentences into account. These models are typically trained using cross-entropy loss in order to maximize the likelihood of the ground-truth …

Content truncated to 500 words