Source-summary Entity Aggregation in Abstractive Summarization

Proceedings of the 29th International Conference on Computational Linguistics, pages 6019–6034 October 12–17, 2022. 6019 Source-summary Entity Aggregation in Abstractive Summarization José Ángel González Valencian Research Institute for Artificial Intelligence, Valencia jogonba2@dsic.upv.es Annie Louis Google Research, London annielouis@google.com Jackie C. K. Cheung McGill University/MILA, Montreal Canada CIFAR AI Chair jcheung@cs.mcgill.ca Abstract In a text, entities mentioned earlier can be re- ferred to in later discourse by a more general description. For example, Celine Dion and Justin Bieber can be referred to by Canadian singers or celebrities. In this work, we study this phenomenon in the context of summariza- tion, where entities from a source text are gen- eralized in the summary. We call such instances source-summary entity aggregations. We cat- egorize these aggregations into two types and analyze them in the CNN/DAILYMAIL corpus, showing that they are reasonably frequent. We then examine how well three state-of-the-art summarization systems can generate such ag- gregations within summaries. We also develop techniques to encourage them to generate more aggregations. Our results show that there is significant room for improvement in producing semantically correct aggregations. 1 Introduction The quality of abstractive summarization systems has improved substantially in the past few years. An important next research question is to better understand the specific linguistic and semantic op- erations which can lead to high-quality abstractive text. In this work, we focus on how entities can be referred to in summaries, especially with an expression more general than in the source. For ex- ample, Table 1 demonstrates how three comic book characters mentioned in the source document are aggregated in a reference summary by the expres- sion, “three of the most well-known comic book characters of all time”. Such referring expressions are interesting for abstractive summarization, since they are novel summary n-grams that result from semantic inference from the source. There are few existing studies about the seman- tics of text generated by current abstractive systems. Some have focused on summary n-grams that are not found in the source text (Kry´sci´nski et al., 2018; Song et al., 2020), and others that look at problems Document (CNN) Comic books of the past few years have seen a lot of changes (a female Thor, anyone?) but not quite so many at one time. Three major characters – Superman, Wonder Woman (both of DC Comics, a Time Warner company, like CNN) and Archie Andrews – came out with new looks (...) Summary Superman, Wonder Woman and Archie all de- buted new looks Thursday. Three of the most well-known comic book characters of all time look radically different. Table 1: An example of source-summary entity aggre- gation. The aggregation “Three of the most well-known comic book characters of all time” is used in the sum- mary to aggregate the entities “Superman”, “Wonder Woman”, and “Archie Andrews” named in the document. resulting from undesirable summary content, e.g., hallucinations (Maynez et al., 2020; Kry´sci´nski et al., 2020). We focus on a specific semantic operation that summary writers can perform in order to change …

Content truncated to 500 words