ASPECTNEWS: Aspect-Oriented Summarization of News Documents

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 6494 - 6506 May 22-27, 2022 c⃝2022 Association for Computational Linguistics ASPECTNEWS: Aspect-Oriented Summarization of News Documents Ojas Ahuja1, Jiacheng Xu1, Akshay Gupta1, Kevin Horecka2, Greg Durrett1 1The University of Texas at Austin 2Walmart NexTech {ojas,jcxu}@utexas.edu, gdurrett@cs.utexas.edu Abstract Generic summaries try to cover an entire doc- ument and query-based summaries try to an- swer document-specific questions. But real users’ needs often fall in between these ex- tremes and correspond to aspects, high-level topics discussed among similar types of docu- ments. In this paper, we collect a dataset of realistic aspect-oriented summaries, ASPECT- NEWS, which covers different subtopics about articles in news sub-domains. We annotate data across two domains of articles, earth- quakes and fraud investigations, where each ar- ticle is annotated with two distinct summaries focusing on different aspects for each domain. A system producing a single generic summary cannot concisely satisfy both aspects. Our fo- cus in evaluation is how well existing tech- niques can generalize to these domains with- out seeing in-domain training data, so we turn to techniques to construct synthetic training data that have been used in query-focused sum- marization work. We compare several train- ing schemes that differ in how strongly key- words are used and how oracle summaries are extracted. Our evaluation shows that our final approach yields (a) focused summaries, better than those from a generic summarization sys- tem or from keyword matching; (b) a system sensitive to the choice of keywords.1 1 Introduction Recent progress in text summarization (See et al., 2017; Liu and Lapata, 2019; Zhang et al., 2020a; Lewis et al., 2020) has been supported by the avail- ability of large amounts of supervised data, such as the CNN/Daily Mail and XSum datasets (Hermann et al., 2015; Narayan et al., 2018), which provide a single, generic, topic-agnostic summary. How- ever, a document often contains different aspects (Titov and McDonald, 2008; Woodsend and Lapata, 2012) that might be relevant to different users. For 1Code is available at https://github.com/oja/ aosumm example, a political science researcher studying re- sponses to earthquakes may want a summary with information about government-led recovery efforts and broader social impacts, not a high-level generic summary of what happened. Systems should be able to produce summaries tailored to the diverse information needs of different users. Crucially, these systems should be usable in realistic settings where a user is interested in vague aspects of the document, instead of a highly focused query. In this work, we present a new dataset for eval- uating single-document aspect-oriented extractive summarization which we call ASPECTNEWS. We derive subsets of examples from CNN/Daily Mail following certain topics, namely earthquakes and fraud reports. These domains are special in that the articles within them have several aspects which are repeatedly mentioned across articles and form coherent topics, e.g., impact on human lives of an earthquake. We ask annotators to select sentences relevant to such information needs, which corre- spond to …

Content truncated to 500 words