A New Approach to Overgenerating and Scoring Abstractive Summaries

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1392–1404 June 6–11, 2021. ©2021 Association for Computational Linguistics 1392 A New Approach to Overgenerating and Scoring Abstractive Summaries Kaiqiang Song,1 Bingqing Wang,2 Zhe Feng,2 Fei Liu1 1University of Central Florida, Orlando, FL 2Robert Bosch LLC, Sunnyvale, CA kqsong@knights.ucf.edu, {bingqing.wang,zhe.feng2}@us.bosch.com feiliu@cs.ucf.edu Abstract We propose a new approach to generate multi- ple variants of the target summary with diverse content and varying lengths, then score and se- lect admissible ones according to users’ needs. Abstractive summarizers trained on single ref- erence summaries may struggle to produce out- puts that achieve multiple desirable properties, i.e., capturing the most important information, being faithful to the original, grammatical and fluent. In this paper, we propose a two-staged strategy to generate a diverse set of candidate summaries from the source text in stage one, then score and select admissible ones in stage two. Importantly, our generator gives a precise control over the length of the summary, which is especially well-suited when space is limited. Our selectors are designed to predict the opti- mal summary length and put special emphasis on faithfulness to the original text. Both stages can be effectively trained, optimized and eval- uated. Our experiments on benchmark summa- rization datasets suggest that this paradigm can achieve state-of-the-art performance. 1 Introduction The learning objective of a modern abstractive sum- marizer is to produce system outputs that resemble reference summaries on a word-to-word basis. It does not promote outputs that possess multiple de- sirable properties, i.e., capturing the most impor- tant information, being faithful to the original text, grammatical and fluent, though some of these prop- erties are exhibited by system abstracts as a natural outcome of a learned summarizer (See et al., 2017; Takase et al., 2016; Tan et al., 2017; Chen and Bansal, 2018; Celikyilmaz et al., 2018; Gehrmann et al., 2018; Liu and Lapata, 2019; Lebanoff et al., 2019b; Fabbri et al., 2019; Bražinskas et al., 2020). Without direct optimization of desired properties, system abstracts often change the meaning of the original document or fail to convey the main con- cepts (Kryscinski et al., 2020). •Source Text Police arrested five anti-nuclear protesters Thursday after they sought to disrupt loading of a French Antarctic research and supply vessel, a spokesman for the protesters said. Summary  Police arrest anti-nuclear protesters  Protesters target French research ship  French police arrest five anti-nuclear protesters  Police arrest five anti-nuclear protesters in Antarctica  Police arrest five anti-nuclear protesters at French Antarctic Table 1: Example of alternative summaries generated from the source text. Admissible summaries are marked by . System summaries that fail to preserve the mean- ing of the source input are marked by . In this paper, we propose a new approach to over- generate and select admissible summaries, which allows a summarizer to juggle multiple objectives and strike a good balance between them (Belz and Reiter, 2006). Our approach consists of two …

Content truncated to 500 words