Diversity driven attention model for query-based abstractive summarization

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1063–1072 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1098 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1063–1072 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1098 Diversity driven attention model for query-based abstractive summarization Preksha Nema† Mitesh M. Khapra† Anirban Laha∗† Balaraman Ravindran† †Indian Institute of Technology Madras, India ∗ IBM Research India {preksha,miteshk}@cse.iitm.ac.in anirlaha@in.ibm.com ravi@cse.iitm.ac.in Abstract Abstractive summarization aims to gen- erate a shorter version of the document covering all the salient points in a com- pact and coherent fashion. On the other hand, query-based summarization high- lights those points that are relevant in the context of a given query. The encode- attend-decode paradigm has achieved notable success in machine translation, extractive summarization, dialog sys- tems, etc. But it suffers from the draw- back of generation of repeated phrases. In this work we propose a model for the query-based summarization task based on the encode-attend-decode paradigm with two key additions (i) a query attention model (in addition to doc- ument attention model) which learns to focus on different portions of the query at different time steps (instead of using a static representation for the query) and (ii) a new diversity based attention model which aims to allevi- ate the problem of repeating phrases in the summary. In order to enable the testing of this model we introduce a new query-based summarization dataset building on debatepedia. Our experi- ments show that with these two addi- tions the proposed model clearly out- performs vanilla encode-attend-decode models with a gain of 28% (absolute) in ROUGE-L scores. 1 Introduction Over the past few years neural models based on the encode-attend-decode (Bahdanau et al., 2014) paradigm have shown great success in various natural language generation (NLG) tasks such as machine translation (Bahdanau et al., 2014), abstractive summarization ((Rush et al., 2015),(Nallapati et al., 2016)) dialog (Li et al., 2016), etc. One such NLG problem which has not received enough attention in the past is query based abstractive text summarization where the aim is to generate the summary of a document in the context of a query. In gen- eral, abstractive summarization, aims to cover all the salient points of a document in a compact and coherent fashion. On the other hand, query focused summarization highlights those points that are relevant in the context of the query. Thus given a document on “the super bowl”, the query “How was the half-time show?”, would result in a summary that would not cover the ac- tual game itself. Note that there has been some work on query based extractive summarization in the past where the aim is to simply extract the most salient sentence(s) from a document and treat these as a summary. There is no natural lan- guage generation involved. Since, we were in- terested in abstractive (as opposed to extractive) summarization we created a new …

Content truncated to 500 words