Generic Text Summarization Using Probabilistic Latent Semantic Indexing

Generic Text Summarization Using Probabilistic Latent Semantic Indexing Harendra Bhandari Graduate School of Information Science Nara Institute of Science and Technology Nara 630-0192, Japan harendra-b@is.naist.jp Takahiko Ito Graduate School of Information Science Nara Institute of Science and Technology Nara 630-0192, Japan takahahi-i@is.naist.jp Masashi Shimbo Graduate School of Information Science Nara Institute of Science and Technology Nara 630-0192, Japan shimbo@is.naist.jp Yuji Matsumoto Graduate School of Information Science Nara Institute of Science and Technology Nara 630-0192, Japan matsu@is.naist.jp Abstract This paper presents a strategy to generate ge- neric summary of documents using Probabilistic Latent Semantic Indexing. Generally a docu- ment contains several topics rather than a single one. Summaries created by human beings tend to cover several topics to give the readers an overall idea about the original document. Hence we can expect that a summary containing sen- tences from better part of the topic spectrum should make a better summary. PLSI has proven to be an effective method in topic detec- tion. In this paper we present a method for cre- ating extractive summary of the document by using PLSI to analyze the features of document such as term frequency and graph structure. We also show our results, which was evaluated us- ing ROUGE, and compare the results with other techniques, proposed in the past. 1 Introduction The advent of the Internet has made a wealth of textual data available to everyone. Finding a spe- cific piece of information in this mass of data can be compared with "finding a small needle in a large heap of straw." Search engines do a remarkable job in providing a subset of the original data set which is generally a lot smaller than the original pile of data. However the subset provided by the search engines is still substantial in size. Users need to manually scan through all the information con- tained in the list of results provided by the search engines until the desired information is found. This makes automatic summarization the task of great importance as the users can then just read the summaries and obtain an overview of the document, hence saving a lot of time during the process. Several methods have been proposed in the field of automatic text summarization. In general two approaches have been taken, extract-based summa- rization and abstract-based summarization. While extract-based summarization focuses in finding relevant sentences from the original document and using the exact sentences as a summary, abstract- based summaries may contain the words or phrases not present in the original document (Mani, 1999). The summarization task can also be classified as query-oriented or generic. The query-oriented summary presents text that contains information relevant to the given query, and the generic sum- marization method presents the summary that gives overall sense of the document (Goldstein et al, 1998). In this paper, we will focus on extract-based generic single-document summarization. In the recent years graph based techinques have become very popular in automatic text summariza- 133 tion (Erkan and Radev, 2004), (Mihalcea, 2005). These techniques …

Content truncated to 500 words