BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarization

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2204–2213 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 2204 BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarization Eva Sharma1, Chen Li2, and Lu Wang1 1Khoury College of Computer Sciences, Northeastern University 2Tencent AI Lab 1evasharma@ccs.neu.edu, luwang@ccs.neu.edu 2ailabchenli@tencent.com Abstract Most existing text summarization datasets are compiled from the news domain, where sum- maries have a flattened discourse structure. In such datasets, summary-worthy content of- ten appears in the beginning of input arti- cles. Moreover, large segments from input ar- ticles are present verbatim in their respective summaries. These issues impede the learn- ing and evaluation of systems that can under- stand an article’s global content structure as well as produce abstractive summaries with high compression ratio. In this work, we present a novel dataset, BIGPATENT, consist- ing of 1.3 million records of U.S. patent doc- uments along with human written abstractive summaries. Compared to existing summa- rization datasets, BIGPATENT has the follow- ing properties: i) summaries contain a richer discourse structure with more recurring enti- ties, ii) salient content is evenly distributed in the input, and iii) lesser and shorter extractive fragments are present in the summaries. Fi- nally, we train and evaluate baselines and pop- ular learning models on BIGPATENT to shed light on new challenges and motivate future di- rections for summarization research. 1 Introduction There has been a growing interest in building neural abstractive summarization systems (See et al., 2017; Paulus et al., 2017; Gehrmann et al., 2018a), which requires large-scale datasets with high quality summaries. A number of summariza- tion datasets have been explored so far (Sandhaus, 2008; Napoles et al., 2012; Hermann et al., 2015; Grusky et al., 2018). However, as most of them are acquired from news articles, they share spe- cific characteristics that limit current state-of-the- art models by making them more extractive rather than allowing them to understand input content and generate well-formed informative summaries. Sample CNN/Daily Mail News Summary An explosion rocks a chemical plant in China’s south- eastern Fujian province for the second time in two years. Six were injured after the explosion and are be- ing hospitalized. The explosion was triggered by an oil leak, though local media has not reported any toxic chemical spills. Sample BIGPATENT Summary A shoelace cover incorporating an interchangeable fashion panel for covering the shoelaces of a gym shoe. The shoelace cover is secured to the shoe by a number of straps threaded through slots in the shoelace cover. These straps secured to each side of the gym shoe in- clude a loop and hook material such that the straps can be disengaged and the shoelace cover can be drawn back to expose the shoelaces. . . Figure 1: Sample summaries from CNN/Daily Mail and BIGPATENT. Extractive fragments reused from input are underlined. Repeated entities indicating dis- course structure are highlighted in respective colors. Specifically, in these datasets, the summaries are flattened narratives with a …

Content truncated to 500 words