Extracting Important Sentences with Support Vector Machines

Extracting Important Sentences with Support Vector Machines Tsutomu HIRAO and Hideki ISOZAKI and Eisaku MAEDA NTT Communication Science Laboratories 2-4, Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0237 Japan {hirao,isozaki,maeda}@cslab.kecl.ntt.co.jp Yuji MATSUMOTO Graduate School of Information and Science, Nara Institute of Science and Technology 8516-9, Takayama, Ikoma, Nara 630-0101 Japan matsu@is.aist-nara.ac.jp Abstract Extracting sentences that contain important in- formation from a document is a form of text summarization. The technique is the key to the automatic generation of summaries similar to those written by humans. To achieve such ex- traction, it is important to be able to integrate heterogeneous pieces of information. One ap- proach, parameter tuning by machine learning, has been attracting a lot of attention. This pa- per proposes a method of sentence extraction based on Support Vector Machines (SVMs). To confirm the method’s performance, we conduct experiments that compare our method to three existing methods. Results on the Text Summa- rization Challenge (TSC) corpus show that our method offers the highest accuracy. Moreover, we clarify the different features effective for ex- tracting different document genres. 1 Introduction Extracting important sentences means extract- ing from a document only those sentences that have important information. Since some sen- tences are lost, the result may lack coherence, but important sentence extraction is one of the basic technologies for generating summaries that are useful for humans to browse. There- fore, this technique plays an important role in automatic text summarization. Many researchers have been studied impor- tant sentence extraction since the late 1950’s (Luhn, 1958). Conventional methods focus on sentence features and define significance scores. The features include key words, sentence posi- tion, and certain linguistic clues. Edmundson (1969) and Nobata et al. (2001) have proposed scoring functions to integrate heterogeneous fea- tures. However, we can not tune the parameter values by hand when the number of features is large. When a large quantity of training data is available, tuning can be effectively realized by machine learning. In recent years, machine learning has attracted attention in the field of automatic text summarization. Aone et al. (1998) and Kupiec et al. (1995) employed Bayesian classifiers, Mani et al. (1998), Nomoto et al. (1997), Lin (1999), and Okumura et al. (1999) used decision tree learning. How- ever, most machine learning methods overfit the training data when many features are given. Therefore, we need to select features carefully. Support Vector Machines (SVMs) (Vapnik, 1995) is robust even when the number of features is large. Therefore, SVMs have shown good performance for text categoriza- tion (Joachims, 1998), chunking (Kudo and Matsumoto, 2001), and dependency structure analysis (Kudo and Matsumoto, 2000). In this paper, we present an important sen- tence extraction technique based on SVMs. We verified the technique against the Text Summa- rization Challenge (TSC) (Fukushima and Oku- mura, 2001) corpus. 2 Important Sentence Extraction 2.1based on Support Vector Machines Support Vector Machines (SVMs) SVM is a supervised learning algorithm for 2- class problems. Training data is given by (x1, y1), · · ·, (xu, yu), xj ∈ …

Content truncated to 500 words