Selective Encoding for Abstractive Sentence Summarization

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1095–1104 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1101 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1095–1104 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1101 Selective Encoding for Abstractive Sentence Summarization Qingyu Zhou†∗ Nan Yang‡ Furu Wei‡ Ming Zhou‡ †Harbin Institute of Technology, Harbin, China ‡Microsoft Research, Beijing, China qyzhou@hit.edu.cn {nanya,fuwei,mingzhou}@microsoft.com Abstract We propose a selective encoding model to extend the sequence-to-sequence frame- work for abstractive sentence summariza- tion. It consists of a sentence encoder, a selective gate network, and an atten- tion equipped decoder. The sentence en- coder and decoder are built with recur- rent neural networks. The selective gate network constructs a second level sen- tence representation by controlling the in- formation flow from encoder to decoder. The second level representation is tailored for sentence summarization task, which leads to better performance. We evalu- ate our model on the English Gigaword, DUC 2004 and MSR abstractive sentence summarization datasets. The experimen- tal results show that the proposed selective encoding model outperforms the state-of- the-art baseline models. 1 Introduction Sentence summarization aims to shorten a given sentence and produce a brief summary of it. This is different from document level summarization task since it is hard to apply existing techniques in extractive methods, such as extracting sentence level features and ranking sentences. Early works propose using rule-based methods (Zajic et al., 2007), syntactic tree pruning methods (Knight and Marcu, 2002), statistical machine translation tech- niques (Banko et al., 2000) and so on for this task. We focus on abstractive sentence summarization task in this paper. Recently, neural network models have been ap- plied in this task. Rush et al. (2015) use auto- constructed sentence-headline pairs to train a neu- ∗Contribution during internship at Microsoft Research. ral network summarization model. They use a Convolutional Neural Network (CNN) encoder and feed-forward neural network language model decoder for this task. Chopra et al. (2016) ex- tend their work by replacing the decoder with Re- current Neural Network (RNN). Nallapati et al. (2016) follow this line and change the encoder to RNN to make it a full RNN based sequence-to- sequence model (Sutskever et al., 2014). the sri lankan government on wednesday announced the closure of government schools with immediate effect as a military campaign against tamil separatists escalated in the north of the country . sri lanka closes schools as war escalates Figure 1: An abstractive sentence summarization system may produce the output summary by dis- tilling the salient information from the highlight to generate a fluent sentence. We model the distilling process with selective encoding. All the above works fall into the encoding- decoding paradigm, which first encodes the in- put sentence to an abstract representation and then decodes the intended output sentence based on the encoded information. As an extension of the encoding-decoding framework, attention- based approach (Bahdanau et …

Content truncated to 500 words