Neural Extractive Text Summarization with Syntactic Compression

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3292–3303, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 3292 Neural Extractive Text Summarization with Syntactic Compression Jiacheng Xu and Greg Durrett Department of Computer Science The University of Texas at Austin {jcxu,gdurrett}@cs.utexas.edu Abstract Recent neural network approaches to summa- rization are largely either selection-based ex- traction or generation-based abstraction. In this work, we present a neural model for single-document summarization based on joint extraction and syntactic compression. Our model chooses sentences from the document, identifies possible compressions based on con- stituency parses, and scores those compres- sions with a neural model to produce the fi- nal summary. For learning, we construct or- acle extractive-compressive summaries, then learn both of our components jointly with this supervision. Experimental results on the CNN/Daily Mail and New York Times datasets show that our model achieves strong performance (comparable to state-of-the-art systems) as evaluated by ROUGE. More- over, our approach outperforms an off-the- shelf compression module, and human and manual evaluation shows that our model’s out- put generally remains grammatical. 1 Introduction Neural network approaches to document sum- marization have ranged from purely extractive (Cheng and Lapata, 2016; Nallapati et al., 2017; Narayan et al., 2018) to abstractive (Rush et al., 2015; Nallapati et al., 2016; Chopra et al., 2016; Tan et al., 2017; Gehrmann et al., 2018). Ex- tractive systems are robust and straightforward to use. Abstractive systems are more flexible for var- ied summarization situations (Grusky et al., 2018), but can make factual errors (Cao et al., 2018; Li et al., 2018) or fall back on extraction in practice (See et al., 2017). Extractive and compressive sys- tems (Berg-Kirkpatrick et al., 2011; Qian and Liu, 2013; Durrett et al., 2016) combine the strengths of both approaches; however, there has been lit- tle work studying neural network models in this vein, and the approaches that have been employed Compression Module Extraction Module Input Document Output Summary … Ref Figure 1: Diagram of the proposed model. Extraction and compression are modularized but jointly trained with supervision derived from the reference summary. typically use seq2seq-based sentence compression (Chen and Bansal, 2018). In this work, we propose a model that can com- bine the high performance of neural extractive systems, additional flexibility from compression, and interpretability given by having discrete com- pression options. Our model first encodes the source document and its sentences and then se- quentially selects a set of sentences to further com- press. Each sentence has a set of compression op- tions available that are selected to preserve mean- ing and grammaticality; these are derived from syntactic constituency parses and represent an ex- panded set of discrete options from prior work (Berg-Kirkpatrick et al., 2011; Wang et al., 2013). The neural model additionally scores and chooses which compressions to apply given the context of the document, the sentence, and the decoder model’s recurrent state. A principal challenge …

Content truncated to 500 words