Enhancing Extractive Text Summarization with Topic-Aware Graph Neural Networks

Proceedings of the 28th International Conference on Computational Linguistics, pages 5360–5371 Barcelona, Spain (Online), December 8-13, 2020 5360 Enhancing Extractive Text Summarization with Topic-Aware Graph Neural Networks Peng Cui, Le Hu, and Yuanchao Liu* School of Computer Science and Technology, Harbin Institute of Technology Harbin 150001, China {pcui, lhu, lyc}@insun.hit.edu.cn Abstract Text summarization aims to compress a textual document to a short summary while keeping salient information. Extractive approaches are widely used in text summarization because of their fluency and efficiency. However, most of existing extractive models hardly capture inter- sentence relationships, particularly in long documents. They also often ignore the effect of topical information on capturing important contents. To address these issues, this paper proposes a graph neural network (GNN)-based extractive summarization model, enabling to capture inter- sentence relationships efficiently via graph-structured document representation. Moreover, our model integrates a joint neural topic model (NTM) to discover latent topics, which can provide document-level features for sentence selection. The experimental results demonstrate that our model not only substantially achieves state-of-the-art results on CNN/DM and NYT datasets but also considerably outperforms existing approaches on scientific paper datasets consisting of much longer documents, indicating its better robustness in document genres and lengths. Further discussions show that topical information can help the model preselect salient contents from an entire document, which interprets its effectiveness in long document summarization. 1 Introduction Text summarization is an important task in natural language processing, which can help people rapidly acquire important information from a large sum of documents. Previous summarization approaches can be mainly classified into two categories, which are abstractive and extractive. Neural-based abstractive models usually use a seq2seq framework (Sutskever et al., 2014) to generate a word-by-word summary after encoding a full document. By contrast, extractive models directly select important sentences from the original document and then aggregate them into a summary. Abstractive models are generally more flexible but may produce disfluent or ungrammatical summary texts (Liu and Lapata, 2019b), whereas extractive models have advantages in factuality and efficiency (Cao et al., 2018). Despite their success, modeling long-range inter-sentence relationships for summarization remains a challenge (Xu et al., 2019b). Hierarchical networks are usually applied to tackle this problem by modeling a document as a sequence of sequences (Cohan et al., 2018; Zhang et al., 2019). However, empirical observations (Liu and Lapata, 2019a) showed that the use of such a paradigm to model inter- sentence relationships does not provide much performance gain for summarization. Hierarchical approaches are also slow to train and tend to overfit (Xiao and Carenini, 2019). Most recently, Graph Neural Networks (GNNs) are widely explored to model cross-sentence relationships for summarization task. The critical step of this framework is to build an effective document graph. Several studies (Xu et al., 2019a; Yasunaga et al., 2017) built document graphs based on discourse analysis. However, this approach depends on external tools and may lead to other problems, such as semantically fragmented output (Liu et al., 2019). Wang and Liu (2020) built a word–sentence document graph based …

Content truncated to 500 words