Fear the REAPER: A System for Automatic Multi-Document Summarization with Reinforcement Learning

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 681–690, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Fear the REAPER: A System for Automatic Multi-Document Summarization with Reinforcement Learning Cody Rioux University of Lethbridge Lethbridge, AB, Canada cody.rioux@uleth.ca Sadid A. Hasan Philips Research North America Briarcliff Manor, NY, USA sadid.hasan@philips.com Yllias Chali University of Lethbridge Lethbridge, AB, Canada chali@cs.uleth.ca Abstract This paper explores alternate algorithms, reward functions and feature sets for per- forming multi-document summarization using reinforcement learning with a high focus on reproducibility. We show that ROUGE results can be improved using a unigram and bigram similarity metric when training a learner to select sentences for summarization. Learners are trained to summarize document clusters based on various algorithms and reward functions and then evaluated using ROUGE. Our ex- periments show a statistically significant improvement of 1.33%, 1.58%, and 2.25% for ROUGE-1, ROUGE-2 and ROUGE- L scores, respectively, when compared with the performance of the state of the art in automatic summarization with re- inforcement learning on the DUC2004 dataset. Furthermore query focused exten- sions of our approach show an improve- ment of 1.37% and 2.31% for ROUGE-2 and ROUGE-SU4 respectively over query focused extensions of the state of the art with reinforcement learning on the DUC2006 dataset. 1 Introduction The multi-document summarization problem has received much attention recently (Lyngbaek, 2013; Sood, 2013; Qian and Liu, 2013) due to its ability to reduce large quantities of text to a human processable amount as well as its appli- cation in other fields such as question answering (Liu et al., 2008; Chali et al., 2009a; Chali et al., 2009b; Chali et al., 2011b). We expect this trend to further increase as the amount of linguistic data on the web from sources such as social media, wikipedia, and online newswire increases. This paper focuses specifically on utilizing reinforce- ment learning (Sutton and Barto, 1998; Szepesv, 2009) to create a policy for summarizing clusters of multiple documents related to the same topic. The task of extractive automated multi- document summarization (Mani, 2001) is to se- lect a subset of textual units, in this case sentences, from the source document cluster to form a sum- mary of the cluster in question. This extractive approach allows the learner to construct a sum- mary without concern for the linguistic quality of the sentences generated, as the source documents are assumed to be of a certain linguistic quality. This paper aims to expand on the techniques used in Ryang and Abekawa (2012) which uses a re- inforcement learner, specifically TD(λ), to create summaries of document clusters. We achieve this through introducing a new algorithm, varying the feature space and utilizing alternate reward func- tions. The TD(λ) learner used in Ryang and Abekawa (2012) is a very early reinforcement learning implementation. We explore the option of leveraging more recent research in reinforcement learning algorithms to improve results. To this end we explore the use of SARSA which is a deriva- tive …

Content truncated to 500 words