Inducing Document Structure for Aspect-based Summarization

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6263–6273 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 6263 Inducing Document Structure for Aspect-based Summarization Lea Frermann Amazon Research uedi@frermann.de Alexandre Klementiev Amazon Research klementa@amazon.com Abstract Automatic summarization is typically treated as a 1-to-1 mapping from document to sum- mary. Documents such as news articles, how- ever, are structured and often cover multiple topics or aspects; and readers may be inter- ested in only some of them. We tackle the task of aspect-based summarization, where, given a document and a target aspect, our models generate a summary centered around the as- pect. We induce latent document structure jointly with an abstractive summarization ob- jective, and train our models in a scalable syn- thetic setup. In addition to improvements in summarization over topic-agnostic baselines, we demonstrate the benefit of the learnt doc- ument structure: we show that our models (a) learn to accurately segment documents by aspect; (b) can leverage the structure to pro- duce both abstractive and extractive aspect- based summaries; and (c) that structure is par- ticularly advantageous for summarizing long documents. All results transfer from synthetic training documents to natural news articles from CNN/Daily Mail and RCV1. 1 Introduction Abstractive summarization systems typically treat documents as unstructured, and generate a sin- gle generic summary per document (Rush et al., 2015; Nallapati et al., 2016; See et al., 2017). In this work we argue that incorporating document structure into abstractive summarization systems is beneficial for at least three reasons. First, the induced structure increases model interpretability, and can be leveraged for other purposes such as document segmentation. Second, structure-aware models help alleviate performance bottlenecks as- sociated with summarization of long documents by learning to focus only on the segments relevant to the topic of interest. Third, they can adapt more flexibly to demands of a user who, faced with a long document or a document collection, might be interested only in some of its topics. For example given a set of reviews of a smart- phone, one user might be interested in a summary of opinions on battery life while another may care more about its camera quality; or, given a news article about a body builder running for governor, a reader might care about the effect on his sports career, or on the political consequences (cf., Figure 1 (bottom) for another example). Throughout this paper, we will refer to such topics or perspectives collectively as aspects. We develop models for aspect-based summariza- tion: given a document and a target aspect, our systems generate a summary specific to the aspect. We extend recent neural models (See et al., 2017) for abstractive summarization making the following contributions: • We propose and compare models for aspect- based summarization incorporating different aspect-driven attention mechanisms in both the encoder and the decoder. • We propose a scalable synthetic training setup and show that our models generalize from synthetic to natural documents, side- stepping the …

Content truncated to 500 words