In Conclusion Not Repetition: Comprehensive Abstractive Summarization with Diversified Attention Based on Determinantal Point Processes

In Conclusion Not Repetition: Comprehensive Abstractive Summarization With Diversified Attention Based On Determinantal Point Processes Lei Li1, Wei Liu1, Marina Litvak2, Natalia Vanetik2 and Zuying Huang1 1 Beijing University of Posts and Telecommunications {leili, thinkwee, zoehuang}@bupt.edu.cn 2 Shamoon College of Engineering litvak.marina@gmail.com natalyav@sce.ac.il Abstract Various Seq2Seq learning models designed for machine translation were applied for ab- stractive summarization task recently. Despite these models provide high ROUGE scores, they are limited to generate comprehensive summaries with a high level of abstraction due to its degenerated attention distribution. We introduce Diverse Convolutional Seq2Seq Model(DivCNN Seq2Seq) using Determinan- tal Point Processes methods(Micro DPPs and Macro DPPs) to produce attention distribution considering both quality and diversity. With- out breaking the end to end architecture, Di- vCNN Seq2Seq achieves a higher level of comprehensiveness compared to vanilla mod- els and strong baselines. All the reproducible codes and datasets are available online1. 1 Introduction Given an article, abstractive summarization aims at generating one or several short sentences that cover the main idea of original article, which is a combination of Natural Language Under- standing(NLU) and Natural Language Genera- tion(NLG). Abstractive summarization uses Seq2Seq mod- els (Sutskever et al., 2014) which consist of an en- coder, a decoder and attention mechanism (Mnih et al., 2014). With attention mechanism the de- coder can choose a weighted context representa- tion at each generation step so it can focus on different parts of encoded information. Seq2Seq with attention achieved remarkable results on ma- chine translation (Bahdanau et al., 2014) and other text generation tasks such as abstractive summa- rizaiton (Rush et al., 2015). Unlike machine translation that emphasizes at- tention mechanism as a method of learning word level alignments between source text and target 1available at https://github.com/thinkwee/DPP CNN Sum marization Article: marseille , france the french prosecutor lead- ing an investigation into the crash of germanwings flight 9525 insisted wednesday that he was not aware of any video footage from on board the plane . marseille pros- ecutor brice robin told cnn that so far no videos were used in the crash investigation ...... of a cell phone video showing the harrowing final seconds from on board ger- manwings flight 9525 as it crashed into the french alps . ......paris match and bild reported that the video was re- covered from a phone at the wreckage site . ...... cnn ’s frederik pleitgen , pamela boykoff , antonia mortensen , sandrine amiel and anna-maja rappard contributed to this report . CNN Seq2Seq: french prosecutor UNK robin says he was not aware of any video . DivCNN Seq2Seq with Micro DPPs: new french pros- ecutor leading an investigation into the crash of UNK wings flight UNK 25 which crashed into french alps . the video was recovered from a phone at the wreckage site . DivCNN Seq2Seq with Macro DPPs: french prosecutor says he was not aware of any video footage from on board UNK wings flight UNK 25 as it crashed into french alps . Table 1: Article-summary sample from …

Content truncated to 500 words