Abstractive Document Summarization with a Graph-Based Attentional Neural Model

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1171–1181 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1108 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1171–1181 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1108 Abstractive Document Summarization with a Graph-Based Attentional Neural Model Jiwei Tan, Xiaojun Wan and Jianguo Xiao Institute of Computer Science and Technology, Peking University The MOE Key Laboratory of Computational Linguistics, Peking University {tanjiwei,wanxiaojun,xiaojianguo}@pku.edu.cn Abstract Abstractive summarization is the ultimate goal of document summarization research, but previously it is less investigated due to the immaturity of text generation tech- niques. Recently impressive progress has been made to abstractive sentence sum- marization using neural models. Unfor- tunately, attempts on abstractive docu- ment summarization are still in a primi- tive stage, and the evaluation results are worse than extractive methods on bench- mark datasets. In this paper, we review the difficulties of neural abstractive docu- ment summarization, and propose a novel graph-based attention mechanism in the sequence-to-sequence framework. The in- tuition is to address the saliency factor of summarization, which has been over- looked by prior works. Experimental re- sults demonstrate our model is able to achieve considerable improvement over previous neural abstractive models. The data-driven neural abstractive method is also competitive with state-of-the-art ex- tractive methods. 1 Introduction Document summarization is a task to generate a fluent, condensed summary for a document, and keep important information. As a useful tech- nique to alleviate the information overload peo- ple are facing today, document summarization has been extensively investigated. Efforts on docu- ment summarization can be categorized to extrac- tive and abstractive methods. Extractive methods produce the summary of a document by extracting sentences from the original document. They have the advantage of producing fluent sentences and preserving the meaning of original documents, but also inevitably face the drawbacks of information redundancy and incoherence between sentences. Moreover, extraction is far from the way humans write summaries. On the contrary, abstractive methods are able to generate better summaries with the use of ar- bitrary words and expressions, but generating ab- stractive summaries is much more difficult in prac- tice. Abstractive summarization involves sophis- ticated techniques including meaning representa- tion, content organization, and surface realization. Each of these techniques has large space to be im- proved (Yao et al., 2017). Due to the immaturity of natural language generation techniques, fully ab- stractive approaches are still at the beginning and cannot always ensure grammatical abstracts. Recent neural networks enable an end-to-end framework for natural language generation. Suc- cess has been witnessed on tasks like machine translation and image captioning, together with the abstractive sentence summarization (Rush et al., 2015). Unfortunately, the extension of sen- tence abstractive methods to the document sum- marization task is not straightforward. Encoding and decoding for a long sequence of multiple sen- tences, currently still lack satisfactory solutions (Yao et al., 2017). Recent abstractive document summarization …

Content truncated to 500 words