Trainable, Scalable Summarization Using Robust NLP and Machine Learning

Trainable, Scalable Summarization Using Robust NLP and Machine Learning* Chinatsu Aone~, Mary Ellen Okurowski +, Jalnes Gorlinsky~ + tSRA International +l)epartment of Defense 4300 Fair Lakes Court 9800 Savage Road Fairfax, VA 22033 Fort Meade, MD 20755-6000 {aonec, gorlinsk}@sra.com meokuro~_ afterlife.ncsc.mil Abstract We describe a trainable and scalable sum- marization system which utilizes features derived from information retrieval, inibr- mation extraction, and NLP techniques and on-line resources. The system con> bines these features using a trainable fea- ture combiner learned from summary ex- amples through a machine learning algo- rithm. We demonstrate system scalability by reporting results on the best combina- tion of summarization features for different document sources. We also present prelim- inary results from a task-based evaluation on summarization outpnt usability. 1 Introduction Frequency-based (Edmundson, 196(.); Kupiec, Ped- ersen, and Chen, 1995; Brandow, Mitze. and Rau, 1995), knowledge-based (Reimer and Hahn, 1988; McKeown and Radev, 1995), and discourse- based (Johnson et al., 1993; Miike et al., 1994; Jones, 1995) approaches to automated summarization cor- respond to a continuum of increasing understanding of the text and increasing complexity in text pro- cessing. Given the goal of machine-generated sum- maries, these approaches attempt to answer three central questions: • How does the system count words to calculate worthiness for summarization? • How does the system incorporate the knowledge of the domain represented in tile text? • How does the system create a coherent and co- hesive summary? Our work leverages off of research in these three approaches and attempts to remedy some of the dif- ficulties encountered in each by applying a combina- tion of information retrieval, information extraction, *We would like to thank Ja.mie Callan for his help with the INQUERY experiments. and NLP techniques and on-line resources with nm- chine learning to generate summaries. Our DimSum system follows a common paradigm of sentence ex- traction, but automates acquiring candidate knowl- edge and learns what knowledge is necessary to sun> inarize. We present how we automatically acquire caudi- date features in Section 2. Section 3 describes our training methodology for combining features to gen- erate summaries, and discusses evaluation results of both batch and machine learning methods. Section 4 reports our task-based evalnation. 2 Extracting Features Ill this section, we describe how the sys- tem counts linguistically-motivated, autornatically- derived words and nmlti-words in calculating wor- thiness for smnmarizat.ion. We show how tile sys- tetll uses an external corpus t.o incorporate domain knowledge in contrast to text-only statistics. Fi- nally, we explain how we attempt to increase the co hesiveness of our summaries by using name aliasing, WordNet synonyms, and morphological variants. 2.1 Defining Single and Multi-word Terms Frequency-based summarization systems typically use a single word string as the unit for counting fl'e- quency. Though robust, such a method ignores the semantic content of words and their potential men> bership in multi-word phrases and may introduce noise in frequency counting by treating the same strings uniformly regardless of context. Our approach, similar to (Tzoukerman, Klavans, and aacquemin, 1997), is to …

Content truncated to 500 words