WikiAsp: A Dataset for Multi-domain Aspect-based Summarization

WikiAsp: A Dataset for Multi-domain Aspect-based Summarization Hiroaki Hayashi1 Prashant Budania2 Peng Wang2 Chris Ackerson2 Raj Neervannan2 Graham Neubig1 1Language Technologies Institute, Carnegie Mellon University 2AlphaSense {hiroakih,gneubig}@cs.cmu.edu {pbudania,pwang,cackerson,rneervannan}@alpha-sense.com Abstract Aspect-based summarization is the task of gen- erating focused summaries based on specific points of interest. Such summaries aid efficient analysis of text, such as quickly understanding reviews or opinions from different angles. However, due to large differences in the type of aspects for different domains (e.g., senti- ment, product features), the development of previous models has tended to be domain- specific. In this paper, we propose WikiAsp,1 a large-scale dataset for multi-domain aspect- based summarization that attempts to spur research in the direction of open-domain aspect-based summarization. Specifically, we build the dataset using Wikipedia articles from 20 different domains, using the section titles and boundaries of each article as a proxy for aspect annotation. We propose sev- eral straightforward baseline models for this task and conduct experiments on the dataset. Results highlight key challenges that existing summarization models face in this setting, such as proper pronoun handling of quoted sources and consistent explanation of time-sensitive events. 1 Introduction Aspect-based summarization is a subtask of sum- marization that aims to provide targeted sum- maries of a document from different perspectives (Titov and McDonald, 2008; Lu et al., 2009; Wang and Ling, 2016; Yang et al., 2018; Angelidis and Lapata, 2018). Unlike generic summarization, this gives more concise summaries that are separated according to specific points of interest, allowing readers to fulfill focused information needs more easily and quickly. However, existing aspect- 1http://github.com/neulab/wikiasp. based summarization work is somewhat narrowly focused; for example, a great majority of the work focuses specifically on the domain of pro- duct or restaurant reviews. In contrast, generic summarization models are tested on a much wider variety of genres, from newswire (Nallapati et al., 2016; Grusky et al., 2018), to academic papers (Kang et al., 2018; Kedzie et al., 2018), to movie scripts (Gorinski and Lapata, 2015). For each genre, the types and characteristics of aspects that will need to be touched upon in a good summary will differ greatly. One natural source of such multi-domain ar- ticles is Wikipedia, and the section boundaries and titles in each article form natural annotations of aspects and corresponding text. There have recently been a number of attempts to generate the lead section of Wikipedia articles from the linked external sites in the reference section (Liu et al., 2018; Fan et al., 2019; Liu and Lapata, 2019a), an approach that does not explicitly consider the different aspects covered by the article. Perez- Beltrachini (2019) also examine domain differ- ences in Wikipedia text summarization. However, existing datasets and analyses lack structure, broad domain coverage, or both. We argue that (1) generating structured summaries is of inherent interest, as these will allow humans consuming the information to browse specific aspects of interest more readily, and (2) the structure will vary across domains, with different domains demonstrating very different characteristics. In this paper, we …

Content truncated to 500 words