BASS: Boosting Abstractive Summarization with Unified Semantic Graph

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 6052–6067 August 1–6, 2021. ©2021 Association for Computational Linguistics 6052 BASS: Boosting Abstractive Summarization with Unified Semantic Graph Wenhao Wu1∗, Wei Li2, Xinyan Xiao2, Jiachen Liu2,Ziqiang Cao3, Sujian Li1†, Hua Wu2, Haifeng Wang2 1Key Laboratory of Computational Linguistics, MOE, Peking University 2Baidu Inc., Beijing, China 3Institute of Artificial Intelligence, Soochow University, China {waynewu,lisujian}@pku.edu.cn {liwei85,xiaoxinyan,liujiachen,wu hua,wanghaifeng}@baidu.com {zqcao}@suda.edu.cn Abstract Abstractive summarization for long-document or multi-document remains challenging for the Seq2Seq architecture, as Seq2Seq is not good at analyzing long-distance relations in text. In this paper, we present BASS, a novel framework for Boosting Abstractive Summa- rization based on a unified Semantic graph, which aggregates co-referent phrases distribut- ing across a long range of context and con- veys rich relations between phrases. Further, a graph-based encoder-decoder model is pro- posed to improve both the document repre- sentation and summary generation process by leveraging the graph structure. Specifically, several graph augmentation methods are de- signed to encode both the explicit and im- plicit relations in the text while the graph- propagation attention mechanism is developed in the decoder to select salient content into the summary. Empirical results show that the proposed architecture brings substantial im- provements for both long-document and multi- document summarization tasks. 1 Introduction Nowadays, the sequence-to-sequence (Seq2Seq) based summarization models have gained unprece- dented popularity (Rush et al., 2015; See et al., 2017; Lewis et al., 2020). However, complex sum- marization scenarios such as long-document or multi-document summarization (MDS), still bring great challenges to Seq2Seq models (Cohan et al., 2018; Liu et al., 2018). In a long document nu- merous details and salient content may distribute evenly (Sharma et al., 2019) while multiple doc- uments may contain repeated, redundant or con- tradictory information (Radev, 2000). These prob- lems make Seq2Seq models struggle with content selection and organization which mainly depend ∗Work is done during an internship at Baidu Inc. †Corresponding author. explanation cop nomod:of ���������� Graph Construction Albert Einstein, a German physicist, published the theory of relativity. He won the physics Nobel Prize in 1921. The great prize was for his explanation of the photoelectric effect. 1912 appos dobj obj nsubj The Unified Semantic Graph the photoelectric effect the physics Nobel Prize won the theory of relativity a German physicist Albert Einstein published was for nsubj obl nsubj Human Written Summary Albert Einstein received the physics Nobel Prize in 1912 for his discovery of the law of the photoelectric effect� Figure 1: Illustration of a unified semantic graph and its construction procedure for a document containing three sentences. In Graph Construction, underlined tokens represent phrases., co-referent phrases are represented in the same color. In The Unified Semantic Graph, nodes of different colors indicate different types, ac- cording to section 3.1. on the long source sequence (Shao et al., 2017). Thus, how to exploit deep semantic structure in the complex text input is a key to further promote summarization …

Content truncated to 500 words