Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedback

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 6252–6272 July 9-14, 2023 ©2023 Association for Computational Linguistics Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedback Paul Roitβγ∗ Johan Ferretγ∗ Lior Shaniγ∗ Roee Aharoniγ Geoffrey Cideronγ Robert Dadashiγ Matthieu Geistγ Sertan Girginγ Léonard Hussenotγ Orgad Kellerγ Nikola Momchevγ Sabela Ramosγ Piotr Stanczykγ Nino Vieillardγ Olivier Bachemγ Gal Elidanγ Avinatan Hassidimγ Olivier Pietquinγ Idan Szpektorγ βBar-Ilan University γGoogle Research {plroit,jferret,liorshani}@google.com Abstract Despite the seeming success of contemporary grounded text generation systems, they of- ten tend to generate factually inconsistent text with respect to their input. This phenomenon is emphasized in tasks like summarization, in which the generated summaries should be cor- roborated by their source article. In this work we leverage recent progress on textual entail- ment models to directly address this problem for abstractive summarization systems. We use reinforcement learning with reference-free, textual-entailment rewards to optimize for fac- tual consistency and explore the ensuing trade- offs, as improved consistency may come at the cost of less informative or more extractive summaries. Our results, according to both au- tomatic metrics and human evaluation, show that our method considerably improves the faithfulness, salience and conciseness of the generated summaries. 1 Introduction Recent advancements in abstractive summarization systems (Zhang et al., 2019; Liu et al., 2022b) are often impeded by their tendency to output infor- mation that is either contradicting or unsupported by their input article, often termed as “hallucina- tions” or factual inconsistency (Falke et al., 2019; Maynez et al., 2020; Pagnoni et al., 2021). While these systems produce highly relevant and coherent text, this lack of factual consistency often limits their wide-spread adoption in real-world applica- tions. An example is depicted in Figure 1, where the highlighted statement in the summary, while plausible, has no support in the input article. ∗Equal contribution Figure 1: Summaries produced by multiple methods from a news article in the XSum dataset. Hallucina- tions or contradictions are highlighted in red. Note how the T5 generated summary mentions that there is a fall in operating profits for the second year in a row, while the article only discusses a recent decline in earnings and a warning made in the previous year. Since widely-used metrics such as ROUGE (Lin, 2004) were shown to be inefficient for detecting hallucinations, many recent research efforts intro- duced novel automatic metrics for measuring fac- tual consistency (Kryscinski et al., 2020; Goyal and Durrett, 2020; Scialom et al., 2021, inter alia). We propose to leverage these automatic metrics within a reinforcement learning (RL) framework at train- ing time. Specifically, we apply textual entailment assessment (a.k.a. natural language inference, or NLI; Dagan et al., 2005; Bowman et al., 2015) be- tween the source article and the generated summary as a reward. Our reward is based on the well studied textual entailment task (Pavlick and Kwiatkowski, 2019; McCoy et al., 2019; MacCartney and Manning, 2007, inter alia), for which there are many pub- licly available …

Content truncated to 500 words