A Text-Extraction Based Summarizer

A Text-Extraction Based Summarizer Tomek Strzalkowski, Gees C. Stein and G. GE Corporate Research & Development 1 Research Circle Niskayuna, NY 12309, USA Bowden Wise Abstract We present an automated method of generating human-readable summaries from a variety of text documents including newspaper articles, business reports, government documents, even broadcast news transcripts. Our approach exploits an em- pirical observation that much of the written text display certain regularities of organization and style, which we call the Discourse Macro Structure (DMS). A summary is therefore created to reflect the components of a given DMS. In order to pro- duce a coherent and readable summary we select continuous, well-formed passages from the source document and assemble them into a mini-document within a DMS template. In this paper we describe an automated summarizer that can generate both short indicative abstracts, useful for quick scanning of a list of documents, as well as longer informative digests that can serve as surrogates for the full text. The summarizer can assist the users of an informa- tion retrieval system in assessing the quality of the results returned from a search, preparing reports and memos for their customers, and even building more effective search queries. Introduction A good summarization tool can be of enormous help for those who have to process large amounts of doc- uments. In information retrieval one would bene- fit greatly from having content-indicative quick-read summaries supplied along with the titles returned from search. Similarly, application areas like rout- ing, news on demand, market intelligence and topic tracking would benefit from a good summarization tool. Perhaps the most difficult problem in designing an automatic text summarization is to define what a summary is, and how to tell a summary from a non-summary, or a good summary from a bad one. The answer depends in part upon who the summary is intended for, and in part upon what it is meant to achieve, which in large measure precludes any objec- tive evaluation. A good summary should at least be a good reflection of the original document while be- ing considerably shorter than the original thus sav- ing the reader valuable reading time. In this paper we describe an automatic way to generate summaries from text-only documents. The summarizer we developed can create general and topical indicative summaries, and also topical in- formative summaries. Our approach is domain- independent and takes advantage of certain organi- zation regularities that were observed in news-type documents. The system participated in a third- party evaluation program and turned out to be one of the top-performing summarizers. Especially the quality/length ratio was very good since our sum- maries tend to be very short (10% of the original length). The summarizer is still undergoing improvement and expansion in order to be able to summarize a wide variety of documents. It is also used success- fully as a tool to solve different problems, like infor- mation retrieval and topic tracking. Task Description and Related Work For most of us, a summary is a …

Content truncated to 500 words