Surfer100: Generating Surveys From Web Resources, Wikipedia-style

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 5388–5392 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 5388 Surfer100: Generating Surveys From Web Resources, Wikipedia-style Irene Li, Alexander Fabbri, Rina Kawamura, Yixin Liu, Xiangru Tang, Jaesung Tae, Chang Shen, Sally Ma, Tomoe Mizutani, Dragomir Radev Yale University {irene.li,alexander.fabbri,rina.kawamura,yixin.liu,xiangru.tang,jake.tae}@yale.edu {chang.shen,sally.ma,tomoe.mizutani,dragomir.radev}@yale.edu Abstract Fast-developing fields such as Artificial Intelligence (AI) often outpace the efforts of encyclopedic sources such as Wikipedia, which either do not completely cover recently-introduced topics or lack such content entirely. As a result, methods for automatically producing content are valuable tools to address this information overload. We show that recent advances in pretrained language modeling can be combined for a two-stage extractive and abstractive approach for Wikipedia lead paragraph generation. We extend this approach to generate longer Wikipedia-style summaries with sections and examine how such methods struggle in this application through detailed studies with 100 reference human-collected surveys. This is the first study on utilizing web resources for long Wikipedia-style summaries to the best of our knowledge. Keywords: web resources, survey generation, abstractive summarization 1. Introduction Novel concepts are being introduced and evolving at a rate that makes creating high-quality, up-to-date Wikipedia pages for such topics challenging. pipeline for automatically creating such WikipediaA pages is thus desirable. While there has been some work on generating full Wikipedia pages, these efforts are either domain-specific (Sauper and Barzilay, 2009), making strong assumptions about the topics being sum- marized (Banerjee and Mitra, 2016), or are purely ex- tractive (Jha et al., 2015). In a related line of work, query-based summarization has been applied to spe- cific sections of Wikipedia pages (Deutsch and Roth, 2019; Zhu et al., 2019), which can be viewed as a more self-contained version of Wikipedia page generation. Recent Wikipedia page generation work has focused on generating the initial leading paragraph of a Wikipedia page (Liu et al., 2018; Liu and Lapata, 2019; Perez- Beltrachini et al., 2019). These papers consist of a two- step framework by which an extractive method selects relevant content for a specific topic, and an abstractive method generates the final summary of the topic. In this paper, we first examine how recently-introduced pretrained language models (Devlin et al., 2019; Liu et al., 2019; Lewis et al., 2019) improve upon both the extractive and abstractive steps of previous models for the task of lead paragraph generation. We further fo- cus on analyzing the extension of such methods to full Wikipedia page generation on scientific topics related to AI and Natural Language Processing (NLP). We manually create summaries of 100 AI and NLP topics divided along sections, as on Wikipedia pages. We per- form ablation studies on content selection and genera- tion methods over selected topics, finding that current content selection methods are not precise and fail to differentiate content well among queries for subtopics of the main topic. Our contributions are: 1) We demonstrate how recent advances in pretrained language models improve upon Wikipedia lead paragraph …

Content truncated to 500 words