Compressive Summarization with Plausibility and Salience Modeling

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 6259–6274, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 6259 Compressive Summarization with Plausibility and Salience Modeling Shrey Desai Jiacheng Xu Greg Durrett Department of Computer Science The University of Texas at Austin shreydesai@utexas.edu {jcxu, gdurrett}@cs.utexas.edu Abstract Compressive summarization systems typically rely on a crafted set of syntactic rules to de- termine what spans of possible summary sen- tences can be deleted, then learn a model of what to actually delete by optimizing for con- tent selection (ROUGE). In this work, we pro- pose to relax the rigid syntactic constraints on candidate spans and instead leave compression decisions to two data-driven criteria: plausi- bility and salience. Deleting a span is plau- sible if removing it maintains the grammatical- ity and factuality of a sentence, and spans are salient if they contain important information from the summary. Each of these is judged by a pre-trained Transformer model, and only deletions that are both plausible and not salient can be applied. When integrated into a simple extraction-compression pipeline, our method achieves strong in-domain results on bench- mark summarization datasets, and human eval- uation shows that the plausibility model gener- ally selects for grammatical and factual dele- tions. Furthermore, the flexibility of our ap- proach allows it to generalize cross-domain: our system fine-tuned on only 500 samples from a new domain can match or exceed an in-domain extractive model trained on much more data.1 1 Introduction Compressive summarization systems offer an ap- pealing tradeoff between the robustness of extrac- tive models and the flexibility of abstractive mod- els. Compression has historically been useful in heuristic-driven systems (Knight and Marcu, 2000, 2002; Wang et al., 2013) or in systems with only certain components being learned (Martins and Smith, 2009; Woodsend and Lapata, 2012; Qian and Liu, 2013). End-to-end learning-based com- pressive methods are not straightforward to train: 1Code and datasets available at https://github. com/shreydesai/cups exact derivations of which compressions should be applied are not available, and deriving oracles based on ROUGE (Berg-Kirkpatrick et al., 2011; Durrett et al., 2016; Xu and Durrett, 2019; Mendes et al., 2019) optimizes only for content selection, not grammaticality or factuality of the summary. As a result, past approaches require significant en- gineering, such as creating a highly specific list of syntactic compression rules to identify permissible deletions (Berg-Kirkpatrick et al., 2011; Li et al., 2014; Wang et al., 2013; Xu and Durrett, 2019). Such manually specified, hand-curated rules are fundamentally inflexible and hard to generalize to new domains. In this work, we build a summarization sys- tem that compresses text in a more data-driven way. First, we create a small set of high-recall constituency-based compression rules that cover the space of legal deletions. Critically, these rules are merely used to propose candidate spans, and the ultimate deletion decisions are controlled by two data-driven models capturing different facets of the compression process. Specifically, we model plausibility and salience of span deletions. Plau- sibility is a domain-independent …

Content truncated to 500 words