Extractive Summarization under Strict Length Constraints

Extractive Summarization under Strict Length Constraints Yashar Mehdad1† Kapil Thadani1† Dragomir Radev2 Amanda Stent1 Youssef Billawala1 Karolina Buchner3 1 Yahoo Research 2 University of Michigan 3 Apple {ymehdad, thadani}@yahoo-inc.com, radev@umich.edu, {stent, billawala}@yahoo-inc.com, kbbuchner@gmail.com Abstract In this paper we report a comparison of various techniques for single-document extractive summarization under strict length budgets, which is a common commercial use case (e.g. summarization of news articles by news aggregators). We show that, evaluated using ROUGE, numerous algorithms from the literature fail to beat a simple lead-based baseline for this task. However, a supervised approach with lightweight and efficient features improves over the lead-based baseline. Additional human evaluation demonstrates that the super- vised approach also performs competitively with a commercial system that uses more sophisticated features. Keywords: text summarization, extractive summarization 1. Introduction Extractive text-to-text summarization has a rich history (e.g. (Carbonell and Goldstein, 1998; Erkan and Radev, 2004)). In this task, sentences or text snippets are extracted from an input document or set of documents to produce a summary. However, in previous extractive summariza- tion shared tasks such as the DUC shared tasks (http: //duc.nist.gov/) length restrictions on output sum- maries were quite generous (100 to 400 words, or 400+ characters) as opposed to the very strict length budgets re- quired by current commercial use cases (160 to 300 char- acters for search result and news article summarization, es- pecially on mobile devices). In this work, we focus on a supervised method to generate extractive summaries under strict length constraints without sacrificing meaning or grammaticality. We present evalua- tion results for the single-document news summarization use case, comparing performance on well written articles as well as on a sampling of news articles of random quality. 2. Related Work Research on single-document, extractive summarization has been conducted since the 1950s (Luhn, 1958). Tra- ditionally, extractive single document summarization has focused on scoring, ranking, and extracting the most “in- formative” sentences from a document using various su- pervised (e.g. (Conroy et al., 2004; Daum´e and Marcu, 2006; Lin, 1999; Svore et al., 2007)) and unsupervised (e.g. (Erkan and Radev, 2004; Mei et al., 2010; Mihalcea and Radev, 2011)) methods. Innovations fall into two broad categories: (a) finding ways to assess whether a sentence should be included in a summary; and (b) efficient algo- rithms for exploring the space of possible summaries. A recent study compared a number of extractive summa- rization algorithms (Hong et al., 2014). The best perform- ing algorithm performed global optimization over the input sentence set. However, these algorithms were compared using the DUC 2004 task, (a) which is a multi-document summarization task; and (b) for which the reference sum- maries were abstractive. † Equal contribution. Figure 1: Our summarization system More related to our framework, snippet extraction is a pop- ular approach for search engines and news aggregators to show some content related to the query and the original document (Li et al., 2008). A simple way to identify snip- pets is to extract a passage from specific areas of …

Content truncated to 500 words