Knowledge Graph-Augmented Abstractive Summarization with Semantic-Driven Cloze Reward

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5094–5107 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 5094 Knowledge Graph-Augmented Abstractive Summarization with Semantic-Driven Cloze Reward Luyang Huang1 Lingfei Wu2 and Lu Wang1 1Khoury College of Computer Sciences, Northeastern University, Boston, MA 02115 2IBM Research AI, IBM T.J. Watson Research Center, Yorktown Heights, NY 10598 1luyang.huang96@gmail.com, luwang@ccs.neu.edu 2wuli@us.ibm.com Abstract Sequence-to-sequence models for abstractive summarization have been studied extensively, yet the generated summaries commonly suffer from fabricated content, and are often found to be near-extractive. We argue that, to address these issues, the summarizer should acquire se- mantic interpretation over input, e.g., via struc- tured representation, to allow the generation of more informative summaries. In this pa- per, we present ASGARD, a novel framework for Abstractive Summarization with Graph- Augmentation and semantic-driven RewarD. We propose the use of dual encoders—a sequential document encoder and a graph- structured encoder—to maintain the global context and local characteristics of entities, complementing each other. We further design a reward based on a multiple choice cloze test to drive the model to better capture entity in- teractions. Results show that our models pro- duce significantly higher ROUGE scores than a variant without knowledge graph as input on both New York Times and CNN/Daily Mail datasets. We also obtain better or comparable performance compared to systems that are fine- tuned from large pretrained language models. Human judges further rate our model outputs as more informative and containing fewer un- faithful errors. 1 Introduction Abstractive summarization aims to produce con- cise and informative summaries with the goal of promoting efficient information consumption and knowledge acquisition (Luhn, 1958). Signif- icant progress has been made in this area by de- signing sequence-to-sequence-based neural mod- els for single-document abstractive summariza- tion (Gehrmann et al., 2018; Liu et al., 2018; Liu and Lapata, 2019). However, due to the limita- tions of model structure and word prediction-based Input Article of New York Times: John M. Fabrizi, the mayor of Bridgeport, admitted on Tuesday that he had used cocaine and abused alcohol while in office. Mr. Fabrizi, who was appointed mayor in 2003 after the former mayor, Joseph P. Ganim, went to prison on corruption charges, said he had sought help for his drug problem about 18 months ago and that he had not used drugs since. About four months ago, he added, he stopped drinking alcohol. Constructed Graph: Summary by Human: The Week column. Mayor John Fabrizi of Brigeport, Conn, publicly admits he used cocaine and abused alcohol while in office; says he stopped drinking alcohol and sought help for his drug problem about 18 months ago. cocaine drinking alcohol alcohol John M. Fabrizi, he, ... had used abused stopped Figure 1: Sample knowledge graph constructed from an article snippet. The graph localizes relevant informa- tion for entities (color coded, e.g. “John M. Fabrizi”) or events (underlined) and provides global context. learning objectives, these models frequently pro- duce unfaithful content (Cao et al., 2018) and near- …

Content truncated to 500 words