From Single to Multi-document Summarization

From Single to Multi-document Summarization: A Prototype System and its Evaluation Chin-Yew Lin and Eduard Hovy University of Southern California / Information Sciences Institute 4676 Admiralty Way Marina del Rey, CA 90292 {cyl,hovy}@isi.edu Abstract NeATS is a multi-document summarization system that attempts to extract relevant or interesting portions from a set of documents about some topic and present them in coherent order. NeATS is among the best performers in the large scale summarization evaluation DUC 2001. 1 Introduction In recent years, text summarization has been enjoying a period of revival. Two workshops on Automatic Summarization were held in 2000 and 2001. However, the area is still being fleshed out: most past efforts have focused only on single-document summarization (Mani 2000), and no standard test sets and large scale evaluations have been reported or made available to the English- speaking research community except the TIPSTER SUMMAC Text Summarization evaluation (Mani et al. 1998). To address these issues, the Document Understanding Conference (DUC) sponsored by the National Institute of Standards and Technology (NIST) started in 2001 in the United States. The Text Summarization Challenge (TSC) task under the NTCIR (NII- NACSIS Test Collection for IR Systems) project started in 2000 in Japan. DUC and TSC both aim to compile standard training and test collections that can be shared among researchers and to provide common and large scale evaluations in single and multiple document summarization for their participants. In this paper we describe a multi-document summarization system NeATS. It attempts to extract relevant or interesting portions from a set of documents about some topic and present them in coherent order. We outline the NeATS system and describe how it performs content selection, filtering, and presentation in Section 2. Section 3 gives a brief overview of the evaluation procedure used in DUC-2001 (DUC 2001). Section 4 discusses evaluation metrics, and Section 5 the results. We conclude with future directions. 2 NeATS NeATS is an extraction-based multi-document summarization system. It leverages techniques proved effective in single document summarization such as: term frequency (Luhn 1969), sentence position (Lin and Hovy 1997), stigma words (Edmundson 1969), and a simplified version of MMR (Goldstein et al. 1999) to select and filter content. To improve topic coverage and readability, it uses term clustering, a ‘buddy system’ of paired sentences, and explicit time annotation. Most of the techniques adopted by NeATS are not new. However, applying them in the proper places to summarize multiple documents and evaluating the results on large scale common tasks are new. Given an input of a collection of sets of newspaper articles, NeATS generates summaries in three stages: content selection, filtering, and presentation. We describe each stage in the following sections. 2.1 Content Selection The goal of content selection is to identify important concepts mentioned in a document collection. For example, AA flight 11, AA flight 77, UA flight 173, UA flight 93, New York, World Trade Center, Twin Towers, Osama bin Laden, and al-Qaida are key concepts for a document collection about the September …

Content truncated to 500 words