Cascaded Attention based Unsupervised Information Distillation for Compressive Summarization

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2081–2090 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Cascaded Attention based Unsupervised Information Distillation for Compressive Summarization∗ Piji Li† Wai Lam† Lidong Bing‡ Weiwei Guo§ Hang Li♮ †Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong ‡AI Lab, Tencent Inc., Shenzhen, China §LinkedIn, Mountain View, CA, USA ♮Noah’s Ark Lab, Huawei Technologies, Hong Kong †{pjli, wlam}@se.cuhk.edu.hk, ‡lyndonbing@tencent.com §wguo@linkedin.com, ♮hangli.hl@huawei.com Abstract When people recall and digest what they have read for writing summaries, the im- portant content is more likely to attract their attention. Inspired by this obser- vation, we propose a cascaded attention based unsupervised model to estimate the salience information from the text for compressive multi-document summariza- tion. The attention weights are learned automatically by an unsupervised data re- construction framework which can capture the sentence salience. By adding sparsity constraints on the number of output vec- tors, we can generate condensed informa- tion which can be treated as word salience. Fine-grained and coarse-grained sentence compression strategies are incorporated to produce compressive summaries. Experi- ments on some benchmark data sets show that our framework achieves better results than the state-of-the-art methods. 1 Introduction The goal of Multi-Document Summarization (MDS) is to automatically produce a succinct summary, preserving the most important informa- tion of a set of documents describing a topic1 (Luhn, 1958; Edmundson, 1969; Goldstein et al., 2000; Erkan and Radev, 2004b; Wan et al., 2007; Nenkova and McKeown, 2012). Considering the procedure of summary writing by humans, when people read, they will remember and forget part ∗The work described in this paper is supported by grants from the Research and Development Grant of Huawei Tech- nologies Co. Ltd (YB2015100076/TH1510257) and the Grant Council of the Hong Kong Special Administrative Re- gion, China (Project Code: 14203414). 1A topic represents a real event, e.g., “AlphaGo versus Lee Sedol”. of the content. Information which is more impor- tant may make a deep impression easily. When people recall and digest what they have read to write summaries, the important information usu- ally attracts more attention (the behavioral and cognitive process of selectively concentrating on a discrete aspect of information, whether deemed subjective or objective, while ignoring other per- ceivable information2) since it may repeatedly ap- pears in some documents, or be positioned in the beginning paragraphs. In the context of multi-document summariza- tion, to generate a summary sentence for a key as- pect of the topic, we need to find its relevant parts in the original documents, which may attract more attention. The semantic parts with high atten- tion weights plausibly represent and reconstruct the topic’s main idea. To this end, we propose a cascaded neural attention model to distill salient information from the input documents in an un- supervised data reconstruction manner, which in- cludes two components: reader and recaller. The reader is a gated recurrent neural network (LSTM or GRU) based sentence sequence encoder which can map all the sentences …

Content truncated to 500 words