BoWLer: A neural approach to extractive text summarization

BoWLer: A neural approach to extractive text summarization Pranav Dhakras Manish Shrivastava Language Technologies Research Center, Kohli Center On Intelligent Systems, International Institute of Information Technology, Hyderabad, India pranav.dhakras@research.iiit.ac.in, m.shrivastava@iiit.ac.in Abstract While extractive summarization is a well stud- ied problem, it is far from solved. In recent years a large number of interesting and com- plex models have been used to achieve signif- icant improvements in performance. This can easily be attributed to Deep Learning models and dense vector representations but the per- formance gain comes with the cost of compu- tational and representational complexity. In this work, we present a simple, yet effec- tive approach for extractive summarization of news articles. In line with many recent works in this area we propose an encoder-decoder architecture with a simple bag of word en- coder for sentences followed by an attention based decoder for relevant sentence selection. Our model is trained end-to-end and its per- formance is comparable to the state-of-the-art models while being simpler both in terms of the number of parameters (significantly lesser) as well as the representational complexity. 1 Introduction Single document summarization (SDS) aims to sum- marize a single text document and present the user with the most important aspects of the same. Fur- ther, SDS can be performed in two ways, abstractive and extractive where the former method is aimed at producing denser summaries which appear to be written by a human while the latter picks out key sentences, phrases and words from the original doc- ument and stitches them together to form a coherent summary. Although abstractive summarization nat- urally appears to be more desired of the two, the cur- rent abstractive summarization systems are riddled with problems including limited vocabulary, lack of fluency and most importantly the inability to con- sistently reproduce facts in the same light as in the original document. Extractive summarization on the other hand tends to be less condensed and may not always produce perfectly coherent summaries but produces fluent sentences and presents facts as is due to its property of picking out sentences from the source document. In recent years, several neural summarization sys- tems have been proposed. Notably, (Cheng and La- pata, 2016; Nallapati et al., 2017; Singh et al., 2017; Narayan et al., 2018) present neural extractive sum- marization systems based on encoder-decoder archi- tecture. (Nallapati et al., 2016), (See et al., 2017) and (Paulus et al., 2017) all make modifications to sequence-to-sequence models and show promising results for abstractive summarization. We present, Bag Of Word embeddings LearnER(BOWLER), a simple neural network based approach for extractive summarization. Our approach is completely data driven and trained end-to-end. We conduct experiments on the non- anonymized version of CNN-DailyMail dataset that was originally developed for reading compre- hension tasks by Hermann et al. (2015) and later modified to suit extractive and abstractive sum- marization. The model focuses on the simplicity of representation and moves away from complex compositional representations for input sentences. The intuition is that the semantic content needed …

Content truncated to 500 words