Producing Biographical Summaries: Combining Linguistic Knowledge with Corpus Statistics

Producing Biographical Summaries: Combining Linguistic Knowledge with Corpus Statistics1 Barry Schiffman Columbia University 1214 Amsterdam Avenue New York, NY 10027, USA Bschiff@cs.columbia.edu Inderjeet Mani2 The MITRE Corporation 11493 Sunset Hills Road Reston, VA 20190, USA imani@mitre.org Kristian J. Concepcion The MITRE Corporation 11493 Sunset Hills Road Reston, VA 20190, USA kjc9@mitre.org 1 This work has been funded by DARPA’s Translingual Information Detection, Extraction, and Summarization (TIDES) research program, under contract number DAA-B07-99-C-C201 and ARPA Order H049. 2 Also at the Department of Linguistics, Georgetown University, Washington, D. C. 20037. Abstract We describe a biographical multi- document summarizer that summarizes information about people described in the news. The summarizer uses corpus statistics along with linguistic knowledge to select and merge descriptions of people from a document collection, removing redundant descriptions. The summarization components have been extensively evaluated for coherence, accuracy, and non-redundancy of the descriptions produced. 1 Introduction The explosion of the World Wide Web has brought with it a vast hoard of information, most of it relatively unstructured. This has created a demand for new ways of managing this often unwieldy body of dynamically changing information. The goal of automatic text summarization is to take a partially-structured source text, extract information content from it, and present the most important content in a condensed form in a manner sensitive to the needs of the user and task (Mani and Maybury 1999). Summaries can be ‘generic’, i.e., aimed at a broad audience, or topic-focused, i.e., tailored to the requirements of a particular user or group of users. Multi-Document Summarization (MDS) is, by definition, the extension of single-document summarization to collections of related documents. MDS can potentially help the user to see at a glance what a collection is about, or to examine similarities and differences in the information content in the collection. Specialized multi-document summarization systems can be constructed for various applications; here we discuss a biographical summarizer. Biographies can, of course, be long, as in book-length biographies, or short, as in an author’s description on a book jacket. The nature of descriptions in the biography can vary, from physical characteristics (e.g., for criminal suspects) to scientific or other achievements (e.g., a speaker’s biography). The crucial point here is that facts about a person’s life are selected, organized, and presented so as to meet the compression and task requirements. While book-quality biographies are out of reach of computers, many other kinds can be synthesized by sifting through large quantities of on-line information, a task that is tedious for humans to carry out. We report here on the development of a biographical MDS summarizer that summarizes information about people described in the news. Such a summarizer is of interest, for example, to analysts who want to automatically construct a dossier about a person over time. Rather than determining in advance what sort of information should go into a biography, our approach is more data-driven, relying on discovering how people are actually described in news reports in a collection. We use corpus statistics from …

Content truncated to 500 words