Trainable, Scalable Summarization Using Robust NLP and Machine Learning

Trainable, Scalable Summarization Using Robust NLP and Machine Learning* Chinatsu Aone~, Mary Ellen Okurowski +, James Gorlinsky~ tSRA International +Department of Defense 4300 Fair Lakes Court 9800 Savage Road Fairfax, VA 22033 Fort Meade, MD 20755-6000 {aonec, gorlinsk}@sra.com meokuro@afterlife.ncsc.mil Abstract We describe a trainable and scalable sum- marization system which utilizes features derived from information retrieval, infor- mation extraction, and NLP techniques and on-line resources. The system com- bines these features using a trainable fea- ture combiner learned from summary ex- amples through a machine learning algo- rithm. We demonstrate system scalability by reporting results on the best combina- tion of summarization features for different document sources. We also present prelim- inary results from a task-based evaluation on summarization output usability. 1 Introduction Frequency-based (Edmundson, 1969; Kupiec, Ped- ersen, and Chen, 1995; Brandow, Mitze, and Rau, 1995), knowledge-based (Reimer and Hahn, 1988; McKeown and l:Ladev, 1995), and discourse- based (Johnson et al., 1993; Miike et al., 1994; Jones, 1995) approaches to automated summarization cor- respond to a continuum of increasing understanding of the text and increasing complexity in text pro- cessing. Given the goal of machine-generated sum- maries, these approaches attempt to answer three central questions: • How does the system count words to calculate worthiness for summarization? • How does the system incorporate the knowledge of the domain represented in the text? • How does the system create a coherent and co- hesive summary? Our work leverages off of research in these three approaches and attempts to remedy some of the dif- ficulties encountered in each by applying a combina- tion of information retrieval, information extraction, "We would like to thank Jamie Callan for his help with the INQUERY experiments. and NLP techniques and on-line resources with ma- chine learning to generate summaries. Our DimSum system follows a common paradigm of sentence ex- traction, but automates acquiring candidate knowl- edge and learns what knowledge is necessary to sum- marize. We present how we automatically acquire candi- date features in Section 2. Section 3 describes our training methodology for combining features to gen- erate summaries, and discusses evaluation results of both batch and machine learning methods. Section 4 reports our task-based evaluation. 2 Extracting Features In this section, we describe how the sys- tem counts linguistically-motivated, automatically- derived words and multi-words in calculating wor- thiness for summarization. We show how the sys- tem uses an external corpus to incorporate domain knowledge in contrast to text-only statistics. Fi- nally, we explain how we attempt to increase the co- hesiveness of our summaries by using name aliasing, WordNet synonyms, and morphological variants. 2.1 Defining Single and Multi-word Terms Frequency-based summarization systems typically use a single word string as the unit for counting fre- quency. Though robust, such a method ignores the semantic content of words and their potential mem- bership in multi-word phrases and may introduce noise in frequency counting by treating the same strings uniformly regardless of context. Our approach, similar to (Tzoukerman, Klavans, and Jacquemin, 1997), is to apply NLP …

Content truncated to 500 words