Leveraging Graph to Improve Abstractive Multi-Document Summarization

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6232–6243 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 6232 Leveraging Graph to Improve Abstractive Multi-Document Summarization Wei Li1, Xinyan Xiao1∗, Jiachen Liu1, Hua Wu1, Haifeng Wang1, Junping Du2 1Baidu Inc., Beijing, China 2Beijing University of Posts and Telecommunications {liwei85,xiaoxinyan,liujiachen,wu hua,wanghaifeng}@baidu.com junpingdu@126.com Abstract Graphs that capture relations between textual units have great benefits for detecting salient information from multiple documents and gen- erating overall coherent summaries. In this paper, we develop a neural abstractive multi- document summarization (MDS) model which can leverage well-known graph representa- tions of documents such as similarity graph and discourse graph, to more effectively pro- cess multiple input documents and produce abstractive summaries. Our model utilizes graphs to encode documents in order to cap- ture cross-document relations, which is cru- cial to summarizing long documents. Our model can also take advantage of graphs to guide the summary generation process, which is beneficial for generating coherent and con- cise summaries. Furthermore, pre-trained lan- guage models can be easily combined with our model, which further improve the sum- marization performance significantly. Empir- ical results on the WikiSum and MultiNews dataset show that the proposed architecture brings substantial improvements over several strong baselines. 1 Introduction Multi-document summarization (MDS) brings great challenges to the widely used sequence-to- sequence (Seq2Seq) neural architecture as it re- quires effective representation of multiple input documents and content organization of long sum- maries. For MDS, different documents may contain the same content, include additional information, and present complementary or contradictory infor- mation (Radev, 2000). So different from single doc- ument summarization (SDS), cross-document links are very important in extracting salient informa- tion, detecting redundancy and generating overall coherent summaries for MDS. Graphs that capture ∗Corresponding author. relations between textual units have great benefits to MDS, which can help generate more informa- tive, concise and coherent summaries from multiple documents. Moreover, graphs can be easily con- structed by representing text spans (e.g. sentences, paragraphs etc.) as graph nodes and the semantic links between them as edges. Graph representa- tions of documents such as similarity graph based on lexical similarities (Erkan and Radev, 2004) and discourse graph based on discourse relations (Christensen et al., 2013), have been widely used in traditional graph-based extractive MDS mod- els. However, they are not well studied by most abstractive approaches, especially the end-to-end neural approaches. Few work has studied the effec- tiveness of explicit graph representations on neural abstractive MDS. In this paper, we develop a neural abstractive MDS model which can leverage explicit graph rep- resentations of documents to more effectively pro- cess multiple input documents and distill abstrac- tive summaries. Our model augments the end-to- end neural architecture with the ability to incor- porate well-established graphs into both the docu- ment representation and summary generation pro- cesses. Specifically, a graph-informed attention mechanism is developed to incorporate graphs into the document encoding process, which enables our model to capture richer cross-document relations. Furthermore, graphs are …

Content truncated to 500 words