Enhancing Single-Document Summarization by Combining RankNet and Third-Party Sources

Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 448–457, Prague, June 2007. c⃝2007 Association for Computational Linguistics Enhancing Single-document Summarization by Combining RankNet and Third-party Sources Krysta M. Svore Microsoft Research 1 Microsoft Way Redmond, WA 98052 ksvore@microsoft.com Lucy Vanderwende Microsoft Research 1 Microsoft Way Redmond, WA 98052 Christopher J.C. Burges Microsoft Research 1 Microsoft Way Redmond, WA 98052 Abstract We present a new approach to automatic summarization based on neural nets, called NetSum. We extract a set of features from each sentence that helps identify its impor- tance in the document. We apply novel features based on news search query logs and Wikipedia entities. Using the RankNet learning algorithm, we train a pair-based sentence ranker to score every sentence in the document and identify the most impor- tant sentences. We apply our system to documents gathered from CNN.com, where each document includes highlights and an article. Our system significantly outper- forms the standard baseline in the ROUGE-1 measure on over 70% of our document set. 1 Introduction Automatic summarization was first studied almost 50 years ago by Luhn (Luhn, 1958) and has contin- ued to be a steady subject of research. Automatic summarization refers to the creation of a shortened version of a document or cluster of documents by a machine, see (Mani, 2001) for details. The sum- mary can be an abstraction or extraction. In an ab- stract summary, content from the original document may be paraphrased or generated, whereas in an ex- tract summary, the content is preserved in its original form, i.e., sentences. Both summary types can in- volve sentence compression, but abstracts tend to be more condensed. In this paper, we focus on produc- ing fully automated single-document extract sum- maries of newswire articles. To create an extract, most automatic systems use linguistic and/or statistical methods to identify key words, phrases, and concepts in a sentence or across single or multiple documents. Each sentence is then assigned a score indicating the strength of presence of key words, phrases, and so on. Sentence scoring methods utilize both purely statistical and purely se- mantic features, for example as in (Vanderwende et al., 2006; Nenkova et al., 2006; Yih et al., 2007). Recently, machine learning techniques have been successfully applied to summarization. The meth- ods include binary classifiers (Kupiec et al., 1995), Markov models (Conroy et al., 2004), Bayesian methods (Daum´e III and Marcu, 2005; Aone et al., 1998), and heuristic methods to determine feature weights (Schiffman, 2002; Lin and Hovy, 2002). Graph-based methods have also been employed (Erkan and Radev, 2004a; Erkan and Radev, 2004b; Mihalcea, 2005; Mihalcea and Tarau, 2005; Mihal- cea and Radev, 2006). In 2001–02, the Document Understanding Con- ference (DUC, 2001), issued the task of creat- ing a 100-word summary of a single news article. The best performing systems (Hirao et al., 2002; Lal and Ruger, 2002) used various learning and semantic-based methods, although no system could outperform the baseline with statistical …

Content truncated to 500 words