Graph-based Neural Multi-Document Summarization

Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 452–462, Vancouver, Canada, August 3 - August 4, 2017. c⃝2017 Association for Computational Linguistics Graph-based Neural Multi-Document Summarization Michihiro Yasunaga1 Rui Zhang1 Kshitijh Meelu1 Ayush Pareek2 Krishnan Srinivasan1 Dragomir Radev1 1Department of Computer Science, Yale University 2The LNM Institute of Information Technology {michihiro.yasunaga,r.zhang,kshitijh.meelu}@yale.edu {ayush.original}@gmail.com {krishnan.srinivasan,dragomir.radev}@yale.edu Abstract We propose a neural multi-document sum- marization (MDS) system that incorpo- rates sentence relation graphs. We employ a Graph Convolutional Network (GCN) on the relation graphs, with sentence em- beddings obtained from Recurrent Neural Networks as input node features. Through multiple layer-wise propagation, the GCN generates high-level hidden sentence fea- tures for salience estimation. We then use a greedy heuristic to extract salient sen- tences while avoiding redundancy. In our experiments on DUC 2004, we consider three types of sentence relation graphs and demonstrate the advantage of combin- ing sentence relations in graphs with the representation power of deep neural net- works. Our model improves upon tradi- tional graph-based extractive approaches and the vanilla GRU sequence model with no graph, and it achieves competitive re- sults against other state-of-the-art multi- document summarization systems. 1 Introduction Document summarization aims to produce fluent and coherent summaries covering salient informa- tion in the documents. Many previous summa- rization systems employ an extractive approach by identifying and concatenating the most salient text units (often whole sentences) in the document. Traditional extractive summarizers produce the summary in two steps: sentence ranking and sentence selection. First, they utilize human- engineered features such as sentence position and length (Radev et al., 2004a), word frequency and importance (Nenkova et al., 2006; Hong and Nenkova, 2014), among others, to rank sentence salience. Then, they select summary-worthy sen- tences using a range of algorithms, such as graph centrality (Erkan and Radev, 2004), constraint op- timization via Integer Linear Programming (Mc- Donald, 2007; Gillick and Favre, 2009; Li et al., 2013), or Support Vector Regression (Li et al., 2007) algorithms. Optionally, sentence reordering (Lapata, 2003; Barzilay et al., 2001) can follow to improve coherence of the summary. Recently, thanks to their strong representation power, neural approaches have become popular in text summarization, especially in sentence com- pression (Rush et al., 2015) and single-document summarization (Cheng and Lapata, 2016). Despite their popularity, neural networks still have issues when dealing with multi-document summarization (MDS). In previous neural multi-document sum- marizers (Cao et al., 2015, 2017), all the sentences in the same document cluster are processed inde- pendently. Hence, the relationships between sen- tences and thus the relationships between differ- ent documents are ignored. However, Christensen et al. (2013) demonstrates the importance of con- sidering discourse relations among sentences in multi-document summarization. This work proposes a multi-document summa- rization system that exploits the representational power of deep neural networks and the sentence relation information encoded in graph representa- tions of document clusters. Specifically, we apply Graph Convolutional Networks (Kipf and Welling, 2017) on sentence relation graphs. First, we dis- cuss three different techniques to produce …

Content truncated to 500 words