Generating Summaries with Topic Templates and Structured Convolutional Decoders

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5107–5116 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5107 Generating Summaries with Topic Templates and Structured Convolutional Decoders Laura Perez-Beltrachini Yang Liu Mirella Lapata Institute for Language, Cognition and Computation School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB {lperez,mlap}@inf.ed.ac.uk yang.liu2@ed.ac.uk Abstract Existing neural generation approaches create multi-sentence text as a single sequence. In this paper we propose a structured convo- lutional decoder that is guided by the con- tent structure of target summaries. We com- pare our model with existing sequential de- coders on three data sets representing differ- ent domains. Automatic and human evalua- tion demonstrate that our summaries have bet- ter content coverage. 1 Introduction Abstractive multi-document summarization aims at generating a coherent summary from a cluster of thematically related documents. Recently, Liu et al. (2018) proposed generating the lead sec- tion of a Wikipedia article as a variant of multi- document summarization and released WikiSum, a large-scale summarization dataset which enables the training of neural models. Like most previous work on neural text gen- eration (Gardent et al., 2017; See et al., 2017; Wiseman et al., 2017; Puduppully et al., 2019; Celikyilmaz et al., 2018; Liu et al., 2018; Perez- Beltrachini and Lapata, 2018; Marcheggiani and Perez-Beltrachini, 2018), Liu et al. (2018) rep- resent the target summaries as a single long se- quence, despite the fact that documents are orga- nized into topically coherent text segments, ex- hibiting a specific structure in terms of the con- tent they discuss (Barzilay and Lee, 2004). This is especially the case when generating text within a specific domain where certain topics might be discussed in a specific order (Wray, 2002). For in- stance, the summary in Table 1 is about a species of damselfly; the second sentence describes the re- gion where the species is found and the fourth the type of habitat the species lives in. We would ex- pect other Animal Wikipedia summaries to exhibit similar content organization. In this work we propose a neural model which is guided by the topic structure of target summaries, i.e., the way content is organized into sentences and the type of content these sentences discuss. Our model consists of a structured decoder which is trained to predict a sequence of sentence top- ics that should be discussed in the summary and to generate sentences based on these. We extend the convolutional decoder of Gehring et al. (2017) so as to be aware of which topics to mention in each sentence as well as their position in the target sum- mary. We argue that a decoder which explicitly takes content structure into account could lead to better summaries and alleviate well-known issues with neural generation models being too general, too brief, or simply incorrect. Although content structure has been largely unexplored within neural text generation, it has been been recognized as useful for summariza- tion. Barzilay and Lee (2004) …

Content truncated to 500 words