EntSUM: A Data Set for Entity-Centric Extractive Summarization

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 3355 - 3366 May 22-27, 2022 c⃝2022 Association for Computational Linguistics ENTSUM: A Data Set for Entity-Centric Summarization Mounica Maddela§∗ Georgia Institute of Technology mmadela3@cc.gatech.edu Mayank Kulkarni∗ Bloomberg mkulkarni24@bloomberg.net Daniel Preo¸tiuc-Pietro Bloomberg dpreotiucpie@bloomberg.net Abstract Controllable summarization aims to pro- vide summaries that take into account user- specified aspects and preferences to better as- sist them with their information need, as op- posed to the standard summarization setup which build a single generic summary of a document. We introduce a human-annotated data set (ENTSUM) for controllable summa- rization with a focus on named entities as the aspects to control. We conduct an extensive quantitative analysis to motivate the task of entity-centric summarization and show that ex- isting methods for controllable summarization fail to generate entity-centric summaries. We propose extensions to state-of-the-art summa- rization approaches that achieve substantially better results on our data set. Our analysis and results show the challenging nature of this task and of the proposed data set.12 1 Introduction Automatic summarization is a core NLP problem that aims to extract key information from a large document and present it to the user with the role of assisting them to digest the core information in the document faster and more easily. However, each user may have a distinct information need and generating a single summary for a document is not suitable for all readers of the document. Recently, various setups for summarization were proposed such that user preferences can be taken into account in the summarization process. These include pro- viding guidance signals such as summary length (Kikuchi et al., 2016), allowing users to provide terms of interest such as aspects (Amplayo et al., 2021) or entities (Fan et al., 2018) or providing ∗* Equal Contribution §Work done during an internship at Bloomberg 1The data set is available at: https://zenodo.org/ record/6359875 2The code is available at: https://github.com/ bloomberg/entsum Figure 1: Example of a generic summary (blue), with three entity-centric summaries from ENTSUM focus- ing on the entities in bold. users the flexibility to interact with the summary and explore new facets of interest (Avinesh et al., 2018). The development of such methods may be paramount in enabling the wide-spread usability of summarization technology. Figure 1 shows an example of a document, its generic summary and summaries controlled through salient named enti- ties in the original document. High quality reference data sets are needed to fos- ter development and facilitate benchmarking. Most summarization data sets are obtained using oppor- tunistic methods such as using abstracts written by editors or librarians when indexing documents. These are by default generic, thus not applicable to controllable summarization. Initial research in this area used small scale human annotations to compare between controllable and generic summa- rization methods (Fan et al., 2018; He et al., 2020), but these can be prone to biases or qualitative is- sues, offer only relative quality measurement and do not allow for replicable …

Content truncated to 500 words