Summary Cloze: A New Task for Content Selection in Topic-Focused Summarization

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3720–3729, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 3720 Summary Cloze: A New Task for Content Selection in Topic-Focused Summarization Daniel Deutsch and Dan Roth Department of Computer and Information Science University of Pennsylvania {ddeutsch,danroth}@seas.upenn.edu Abstract A key challenge in topic-focused summariza- tion is determining what information should be included in the summary, a problem known as content selection. In this work, we pro- pose a new method for studying content se- lection in topic-focused summarization called the summary cloze task. The goal of the sum- mary cloze task is to generate the next sen- tence of a summary conditioned on the begin- ning of the summary, a topic, and a reference document(s). The main challenge is deciding what information in the references is relevant to the topic and partial summary and should be included in the summary. Although the cloze task does not address all aspects of the traditional summarization problem, the more narrow scope of the task allows us to collect a large-scale datset of nearly 500k summary cloze instances from Wikipedia. We report ex- perimental results on this new dataset using various extractive models and a two-step ab- stractive model that first extractively selects a small number of sentences and then abstrac- tively summarizes them. Our results show that the topic and partial summary help the models identify relevant content, but the task remains a significant challenge. 1 Introduction Topic-focused multi-document summarization (MDS) has long been a goal for natural language processing (Dang, 2005, 2006). In contrast to generic summarization, topic-focused summa- rization systems attempt to summarize a set of reference documents with respect to a specific topic or information need. Recent research on summarization has mostly focused on generic summarization, in part due to the size of the available datasets. Work on topic-focused summarization (Lin and Bilmes, 2011; Ma et al., 2016; Feigenblat et al., 2017) is … Federer received an honorary doctorate awarded to him by his home university, the University of Basel. He received the title in recognition for his role in increasing the international reputation of Basel and Switzerland, and also his engagement for children in Africa through his charitable foundation. Roger Federer, Philanthropy and outreach The current world number two tennis player, who was born in Basel, received the title from the medical faculty, which praised him for his role in increasing the international reputation of Basel and Switzerland…In a press release it also highlighted the 36-year- old sportsman's engagement for children in Africa through his charitable foundation… topic reference document context cloze input output Figure 1: Given a topic, reference document, and a par- tial summary (the context), the objective of the sum- mary cloze task is to predict the next sentence of the summary, known as the cloze. largely based on the DUC 2005 and 2006 datasets (Dang, 2005, 2006), which are orders …

Content truncated to 500 words