StructSum: Summarization via Structured Representations

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 2575–2585 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 2575 StructSum: Summarization via Structured Representations Vidhisha Balachandran Artidoro Pagnoni Jay Yoon Lee Dheeraj Rajagopal Jaime Carbonell Yulia Tsvetkov School of Computer Science Carnegie Mellon University, Pittsburgh, USA {vbalacha,apagnoni,jaylee,dheeraj,jgc,ytsvetko}@cs.cmu.edu Abstract Abstractive text summarization aims at com- pressing the information of a long source doc- ument into a rephrased, condensed summary. Despite advances in modeling techniques, ab- stractive summarization models still suffer from several key challenges: (i) layout bias: they overfit to the style of training corpora; (ii) limited abstractiveness: they are optimized to copying n-grams from the source rather than generating novel abstractive summaries; (iii) lack of transparency: they are not inter- pretable. In this work, we propose a frame- work based on document-level structure induc- tion for summarization to address these chal- lenges. To this end, we propose incorporating latent and explicit dependencies across sen- tences in the source document into end-to-end single-document summarization models. Our framework complements standard encoder- decoder summarization models by augment- ing them with rich structure-aware document representations based on implicitly learned (latent) structures and externally-derived lin- guistic (explicit) structures. We show that our summarization framework, trained on the CNN/DM dataset, improves the coverage of content in the source documents, generates more abstractive summaries by generating more novel n-grams, and incorporates inter- pretable sentence-level structures, while per- forming on par with standard baselines.1 1 Introduction Text summarization aims at identifying important information in long source documents and express- ing it in human readable summaries. Two promi- nent methods of generating summaries are extrac- tive (Dorr et al., 2003; Nallapati et al., 2017), where important sentences in the source article are se- lected to form a summary, and abstractive (Rush 1Code and data available at: https://github.com/ vidhishanair/structured_summarizer et al., 2015; See et al., 2017), where the model restructures and rephrases essential content into a paraphrased summary. State of the art approaches to abstractive summa- rization employ neural encoder-decoder methods that encode the source document as a sequence of tokens producing latent document representa- tions and decode the summary conditioned on the representations. Recent studies suggest that these models suffer from several key challenges. First, since standard training datasets are derived from news articles, model outputs are strongly affected by the layout bias of the articles, with models rely- ing on the leading sentences of source documents (Kryscinski et al., 2019; Kedzie et al., 2018). Sec- ond, although they aim to generate paraphrased summaries, abstractive summarization systems of- ten copy long sequences from the source, causing their outputs to resemble extractive summaries (Lin and Ng, 2019; Gehrmann et al., 2018). Finally, current methods do not lend themselves easily to interpretation via intermediate structures (Lin and Ng, 2019), which could be useful for identifying major bottlenecks in summarization models. To address these challenges, we introduce Struct- Sum: a framework that incorporates structured doc- ument representations into summarization mod- els. StructSum complements …

Content truncated to 500 words