Significant Sentence Extraction by Euclidean Distance Based on Singular Value Decomposition

Significant Sentence Extraction by Euclidean Distance Based on Singular Value Decomposition Changbeom Lee1, Hyukro Park2, and Cheolyoung Ock1 1 School of Computer Engineering & Information Technology, University of Ulsan, Ulsan 680-749, South Korea {chblee1225, okcy}@mail.ulsan.ac.kr 2 Department of Computer Science, Chonnam National University, 300, Youngbong-dong, Buk-gu, Kwangju 500-757, South Korea hyukro@chonnam.ac.kr Abstract. This paper describes an automatic summarization approach that constructs a summary by extracting the significant sentences. The approach takes advantage of the cooccurrence relationships between terms only in the document. The techniques used are principal compo- nent analysis (PCA) to extract the significant terms and singular value decompostion (SVD) to find out the significant sentences. The PCA can quantify both the term frequency and term-term relationship in the doc- ument by the eigenvalue-eigenvector pairs. And the sentence-term matrix can be decomposed into the proper dimensional sentence-concentrated and term-concentrated marices which are used for the Euclidean dis- tances between the sentence and term vectors and also removed the noise of variability in term usage by the SVD. Experimental results on Korean newspaper articles show that the proposed method is to be preferred over random selection of sentences or only PCA when summarization is the goal. keywords: Text summarization; Principal component analysis; Singular value decomposition. 1 Introduction Automatic text summarization is the process of reducing the length of text documents, while retaining the essential qualities of the orginal. Many search engines have tried to solve the problem of information overflowing by showing either the title and beginning of of a document. However, such the title and beginning are insufficient to decide the relevance of the documents which user wants to search, and this is the reason that the text summarization is required to resolve this problem. The process of text summarization could consist of two phases: a document interpretation phase and a summary generation phase. The primary goal of a document interpretation phase is to find the main theme of a document and its R. Dale et al. (Eds.): IJCNLP 2005, LNAI 3651, pp. 636–645, 2005. c ⃝ Springer-Verlag Berlin Heidelberg 2005 Significant Sentence Extraction by Euclidean Distance 637 corresponding significant words. Since the significant words collectively repre- sent the main theme of a document, it is important to find them more reason- ably. For the purpose of this doing , the word frequency of a document might be utilized[4,8]. But this approach is limited in that cooccurrence relationships among words are not considered at all. In contrast to word frequency, the other method by using WordNet or a thesaurus[2] makes good use of word relationships such as NT(narrow term), RT(related term), and so on. However such resources require a large cost to compile, and often represent too general relationships to fit a specific domain. In this paper, we propose a new summarization approach by both princi- pal component analysis (PCA) and singular value decomposition (SVD) that are called quantification methods or statistical analysis methods. PCA is uti- lized to find significant words or terms in the document by term-relationships. …

Content truncated to 500 words