A Cascade Approach to Neural Abstractive Summarization with Content Selection and Fusion

Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 529–535 December 4 - 7, 2020. c⃝2020 Association for Computational Linguistics 529 A Cascade Approach to Neural Abstractive Summarization with Content Selection and Fusion Logan Lebanoff,♠ Franck Dernoncourt,♦ Doo Soon Kim,♦ Walter Chang,♦ Fei Liu♠ ♠Computer Science Department, University of Central Florida, Orlando, FL ♦Adobe Research, San Jose, CA loganlebanoff@knights.ucf.edu {dernonco,dkim,wachang}@adobe.com feiliu@cs.ucf.edu Abstract We present an empirical study in favor of a cas- cade architecture to neural text summarization. Summarization practices vary widely but few other than news summarization can provide a sufficient amount of training data enough to meet the requirement of end-to-end neural ab- stractive systems which perform content selec- tion and surface realization jointly to generate abstracts. Such systems also pose a challenge to summarization evaluation, as they force con- tent selection to be evaluated along with text generation, yet evaluation of the latter remains an unsolved problem. In this paper, we present empirical results showing that the performance of a cascaded pipeline that separately identifies important content pieces and stitches them to- gether into a coherent text is comparable to or outranks that of end-to-end systems, whereas a pipeline architecture allows for flexible con- tent selection. We finally discuss how we can take advantage of a cascaded pipeline in neu- ral text summarization and shed light on im- portant directions for future research. 1 Introduction There is a variety of successful summarization ap- plications but few can afford to have a large number of annotated examples that are sufficient to meet the requirement of end-to-end neural abstractive summarization. Examples range from summariz- ing radiology reports (Jing et al., 2019; Zhang et al., 2020) to congressional bills (Kornilova and Eidel- man, 2019) and meeting conversations (Mehdad et al., 2013; Li et al., 2019; Koay et al., 2020). The lack of annotated resources suggests that end-to- end systems may not be a “one-size-fits-all” so- lution to neural text summarization. There is an increasing need to develop cascaded architectures to allow for customized content selectors to be com- bined with general-purpose neural text generators to realize the full potential of neural abstractive summarization. We advocate for explicit content selection as it al- lows for a rigorous evaluation and visualization of intermediate results of such a module, rather than associating it with text generation. Existing neu- ral abstractive systems can perform content selec- tion implicitly using end-to-end models (See et al., 2017; Celikyilmaz et al., 2018; Raffel et al., 2019; Lewis et al., 2020), or more explicitly, with an exter- nal module to select important sentences or words to aid generation (Tan et al., 2017; Gehrmann et al., 2018; Chen and Bansal, 2018; Kry´sci´nski et al., 2018; Hsu et al., 2018; Lebanoff et al., 2018, 2019b; Liu and Lapata, 2019). However, content selection concerns not only the selection of important seg- ments from a document, but also the cohesiveness of selected segments and the …

Content truncated to 500 words