Which Scores to Predict in Sentence Regression for Text Summarization?

Proceedings of NAACL-HLT 2018, pages 1782–1791 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Which Scores to Predict in Sentence Regression for Text Summarization? Markus Zopf, Eneldo Loza Menc´ıa, and Johannes F¨urnkranz Research Training Group AIPHES / Knowledge Engineering Group Department of Computer Science, Technische Universit¨at Darmstadt Hochschulstraße 10, 64289 Darmstadt, Germany {zopf@aiphes,eneldo@ke,juffi@ke}.tu-darmstadt.de Abstract The task of automatic text summarization is to generate a short text that summarizes the most important information in a given set of doc- uments. Sentence regression is an emerging branch in automatic text summarizations. Its key idea is to estimate the importance of in- formation via learned utility scores for indi- vidual sentences. These scores are then used for selecting sentences from the source doc- uments, typically according to a greedy se- lection strategy. Recently proposed state-of- the-art models learn to predict ROUGE recall scores of individual sentences, which seems reasonable since the final summaries are evalu- ated according to ROUGE recall. In this paper, we show in extensive experiments that follow- ing this intuition leads to suboptimal results and that learning to predict ROUGE precision scores leads to better results. The crucial dif- ference is to aim not at covering as much infor- mation as possible but at wasting as little space as possible in every greedy step. 1 Introduction More and more data is generated in textual form in newspapers, social media platforms, and micro- blogging services and it has become impossible for humans to read, comprehend, and filter all the available data. Automatic summarization aims at mitigating these problems by “taking an informa- tion source, extracting content from it, and pre- senting the most important content to the user in a condensed form and in a manner sensitive to the users or applications needs” (Mani, 2001). Very prominent in automatic text summariza- tion is the idea of extractive summarization. In ex- tractive summarization, summaries are not gener- ated from scratch. Instead, sentences in the source documents, which are supposed to be summarized, are extracted and concatenated to form a summary. To be able to select sentences in a meaningful manner, it is crucial for the extractive systems to be able to estimate the utility of individual sen- tences. Supervised extractive methods are usually mod- eled in a regression framework. Hence, this sub- field of automatic summarization is called sen- tence regression. The predicted scores are used to generate a ranking of the sentences, and a greedy strategy is often used in combination with addi- tional redundancy avoidance to select sentences which will be added to the iteratively generated summary (Carbonell and Goldstein, 1998). An- other method for the selection is solving an integer linear programming (ILP) problem (Gillick et al., 2008; Hong and Nenkova, 2014) which is, how- ever, an NP-hard problem (Filatova and Hatzivas- siloglou, 2004). Even though it can be argued that the complexity is not an issue since there are good solvers for ILPs, it remains a problem when large document collections with many …

Content truncated to 500 words