SummScreen: A Dataset for Abstractive Screenplay Summarization

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 786 - 807 May 22-27, 2022 c⃝2022 Association for Computational Linguistics HIBRIDS: Attention with Hierarchical Biases for Structure-aware Long Document Summarization Shuyang Cao and Lu Wang Computer Science and Engineering University of Michigan Ann Arbor, MI {caoshuy, wangluxy}@umich.edu Abstract Document structure is critical for efficient in- formation consumption. However, it is chal- lenging to encode it efficiently into the mod- ern Transformer architecture. In this work, we present HIBRIDS, which injects Hierarchical Biases foR Incorporating Document Structure into the calculation of attention scores. We fur- ther present a new task, hierarchical question- summary generation, for summarizing salient content in the source document into a hierar- chy of questions and summaries, where each follow-up question inquires about the content of its parent question-summary pair. We also annotate a new dataset with 6, 153 question- summary hierarchies labeled on long govern- ment reports. Experiment results show that our model produces better question-summary hierarchies than comparisons on both hierar- chy quality and content coverage, a finding also echoed by human judges. Additionally, our model improves the generation of long- form summaries from lengthy government re- ports and Wikipedia articles, as measured by ROUGE scores. 1 Introduction Document structure facilitates information search- ing, reading comprehension, and knowledge ac- quisition by providing an informative overview of the content (Guthrie et al., 1991; Meyer et al., 1980; Taylor and Beach, 1984; Shavelson, 1974; Jonassen, 1988). Specifically, for summarization, its utility is twofold: (1) Source document struc- tures, such as sections and paragraphs, can be in- structive for summary generation (Cohan et al., 2018; Celikyilmaz et al., 2018; Zhang et al., 2019); (2) Structures in output summaries, e.g., time- lines (Shahaf et al., 2012; Wang et al., 2015) or aspects (Angelidis and Lapata, 2018), can also ease content understanding. Nonetheless, state-of-the-art abstractive summa- rization systems, all built on the Transformer archi- tecture (Zhang et al., 2020; Lewis et al., 2020), use attentions to estimate relations between pairwise to- kens and largely ignore document structures. While hierarchical encoding has been investigated (Zhang et al., 2019; Balachandran et al., 2021), its need for training large amounts of additional parameters leads to increased memory footprint and thus limits the allowed input length. As for the output, the structure of single document summaries remains largely “flat”, such as a list of aspects (Meng et al., 2021). We argue that it is imperative to develop systems that can output summaries with rich struc- tures to support knowledge acquisition, which is especially critical for long documents that cover nu- merous subjects with varying details (Huang et al., 2021; Kry´sci´nski et al., 2021). This work consists of two main objectives: (1) effectively informing summarization models of the source document’s structure, and (2) present- ing a new summarization task that produces hi- erarchically organized question-summary pairs to facilitate information consumption. To this end, we propose HIBRIDS (Hierarchical Biases foR Incorporating Document Structure).1 We design learnable hierarchical biases, as …

Content truncated to 500 words