Heterogeneous Graph Neural Networks for Extractive Document Summarization

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6209–6219 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 6209 Heterogeneous Graph Neural Networks for Extractive Document Summarization Danqing Wang∗, Pengfei Liu∗, Yining Zheng, Xipeng Qiu†, Xuanjing Huang Shanghai Key Laboratory of Intelligent Information Processing, Fudan University School of Computer Science, Fudan University 825 Zhangheng Road, Shanghai, China {dqwang18,pfliu14,ynzheng19,xpqiu,xjhuang}@fudan.edu.cn Abstract As a crucial step in extractive document sum- marization, learning cross-sentence relations has been explored by a plethora of approaches. An intuitive way is to put them in the graph- based neural network, which has a more com- plex structure for capturing inter-sentence rela- tionships. In this paper, we present a hetero- geneous graph-based neural network for ex- tractive summarization (HETERSUMGRAPH), which contains semantic nodes of different granularity levels apart from sentences. These additional nodes act as the intermediary be- tween sentences and enrich the cross-sentence relations. Besides, our graph structure is flexible in natural extension from a single- document setting to multi-document via intro- ducing document nodes. To our knowledge, we are the first one to introduce different types of nodes into graph-based neural networks for extractive document summarization and per- form a comprehensive qualitative analysis to investigate their benefits. The code will be re- leased on Github1. 1 Introduction Extractive document summarization aims to extract relevant sentences from the original documents and reorganize them as the summary. Recent years have seen a resounding success in the use of deep neural networks on this task (Cheng and Lapata, 2016; Narayan et al., 2018; Arumae and Liu, 2018; Zhong et al., 2019a; Liu and Lapata, 2019b). These existing models mainly follow the encoder-decoder framework in which each sentence will be encoded by neural components with different forms. To effectively extract the summary-worthy sen- tences from a document, a core step is to model ∗These two authors contributed equally. †Corresponding author. 1https://github.com/brxx122/ HeterSUMGraph the cross-sentence relations. Most current mod- els capture cross-sentence relations with recurrent neural networks (RNNs) (Cheng and Lapata, 2016; Nallapati et al., 2017; Zhou et al., 2018). How- ever, RNNs-based models are usually hard to cap- ture sentence-level long-distance dependency, es- pecially in the case of the long document or multi- documents. One more intuitive way is to model the relations of sentences using the graph struc- ture. Nevertheless, it is challenging to find an ef- fective graph structure for summarization. Efforts have been made in various ways. Early traditional work makes use of inter-sentence cosine similar- ity to build the connectivity graph like LexRank (Erkan and Radev, 2004) and TextRank (Mihalcea and Tarau, 2004). Recently, some works account for discourse inter-sentential relationships when building summarization graphs, such as the Ap- proximate Discourse Graph (ADG) with sentence personalization features (Yasunaga et al., 2017) and Rhetorical Structure Theory (RST) graph (Xu et al., 2019). However, they usually rely on external tools and need to take account of the error propagation problem. A more straightforward way is to create a sentence-level fully-connected graph. To some …

Content truncated to 500 words