Discourse Understanding and Factual Consistency in Abstractive Summarization

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 435–447 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 435 Discourse Understanding and Factual Consistency in Abstractive Summarization Saadia Gabriel♠ Antoine Bosselut♦ Jeff Da ♢ Ari Holtzman♠ Jan Buys♡ Kyle Lo ♢ Asli Celikyilmaz ♣ Yejin Choi♠♢ ♠Paul G. Allen School of Computer Science & Engineering, University of Washington ♣Microsoft Research ♢Allen Institute for Artificial Intelligence ♦Stanford University ♡University of Cape Town {skgabrie,ahai,yejin}@cs.washington.edu, {jeffd,kylel}@allenai.org, antoineb@cs.stanford.edu, jbuys@cs.uct.ac.za, aslicel@microsoft.com Abstract We introduce a general framework for abstrac- tive summarization with factual consistency and distinct modeling of the narrative flow in an output summary. Our work addresses cur- rent limitations of models for abstractive sum- marization that often hallucinate information or generate summaries with coherence issues. To generate abstractive summaries with fac- tual consistency and narrative flow, we pro- pose Cooperative Generator – Discriminator Networks (Co-opNet), a novel transformer- based framework where a generator works with a discriminator architecture to compose coherent long-form summaries. We explore four different discriminator objectives which each capture a different aspect of coherence, including whether salient spans of generated abstracts are hallucinated or appear in the in- put context, and the likelihood of sentence ad- jacency in generated abstracts. We measure the ability of Co-opNet to learn these objectives with arXiv scientific papers, using the abstracts as a proxy for gold long- form scientific article summaries. Empirical results from automatic and human evaluations demonstrate that Co-opNet learns to summa- rize with considerably improved global coher- ence compared to competitive baselines. 1 Introduction Generating summaries with coherent discourse structure and domain knowledge awareness poses a challenge for current methods in summarization. Generative models can commonly produce high quality text (Figure 1), but fail to understand finer- grained details of coherence such as the structure and flow of a narrative. In addition, they often gen- erate factually incorrect content. Prior work on fac- tuality in abstractive summarization has found that current models can hallucinate information more Stochastic birth-death-immigration models are widely used in biology and ecology to study population dynamics. In this paper, we introduce a new formalism for describing... We consider the evolution of multi- species populations in which each individual is assigned a species. The model is based on birth-death- immigration (bdiy), and we assume... We study the evolution of stochastic models for the evolution of multi- species populations, where each species is a clone of a parent species... -13.873 -13.213 -12.883 Figure 1: Generated abstracts for a biology article (from the Bio subset of our arXiv dataset). Abstracts are ranked from most (top) to least likely (bottom) us- ing the generator model. Abstracts with better narra- tive structure and domain-specific content (such as the circled abstract) are often out-ranked in terms of likeli- hood by abstracts with factual errors and less structure. than 70% of the time when generating summaries of news articles (Maynez et al., 2020). To address these issues, we focus our study on generating abstractive summaries …

Content truncated to 500 words