Probabilistic Sentence Reduction Using Support Vector Machines

Probabilistic Sentence Reduction Using Support Vector Machines Minh Le Nguyen, Akira Shimazu, Susumu Horiguchi Bao Tu Ho and Masaru Fukushi Japan Advanced Institute of Science and Technology 1-8, Tatsunokuchi, Ishikawa, 923-1211, JAPAN {nguyenml, shimazu, hori, bao, mfukushi}@jaist.ac.jp Abstract This paper investigates a novel application of sup- port vector machines (SVMs) for sentence reduction. We also propose a new probabilistic sentence reduc- tion method based on support vector machine learn- ing. Experimental results show that the proposed methods outperform earlier methods in term of sen- tence reduction performance. 1 Introduction The most popular methods of sentence reduc- tion for text summarization are corpus based methods. Jing (Jing 00) developed a method to remove extraneous phrases from sentences by using multiple sources of knowledge to de- cide which phrases could be removed. However, while this method exploits a simple model for sentence reduction by using statistics computed from a corpus, a better model can be obtained by using a learning approach. Knight and Marcu (Knight and Marcu 02) proposed a corpus based sentence reduction method using machine learning techniques. They discussed a noisy-channel based approach and a decision tree based approach to sentence reduction. Their algorithms provide the best way to scale up the full problem of sentence re- duction using available data. However, these al- gorithms require that the word order of a given sentence and its reduced sentence are the same. Nguyen and Horiguchi (Nguyen and Horiguchi 03) presented a new sentence reduction tech- nique based on a decision tree model without that constraint. They also indicated that se- mantic information is useful for sentence reduc- tion tasks. The major drawback of previous works on sentence reduction is that those methods are likely to output local optimal results, which may have lower accuracy. This problem is caused by the inherent sentence reduction model; that is, only a single reduced sentence can be obtained. As pointed out by Lin (Lin 03), the best sen- tence reduction output for a single sentence is not approximately best for text summarization. This means that “local optimal” refers to the best reduced output for a single sentence, while the best reduced output for the whole text is “global optimal”. Thus, it would be very valu- able if the sentence reduction task could gener- ate multiple reduced outputs and select the best one using the whole text document. However, such a sentence reduction method has not yet been proposed. Support Vector Machines (Vapnik 95), on the other hand, are strong learning methods in com- parison with decision tree learning and other learning methods (Sholkopf 97). The goal of this paper is to illustrate the potential of SVMs for enhancing the accuracy of sentence reduc- tion in comparison with previous work. Accord- ingly, we describe a novel deterministic method for sentence reduction using SVMs and a two- stage method using pairwise coupling (Hastie 98). To solve the problem of generating mul- tiple best outputs, we propose a probabilistic sentence reduction model, in which a variant of probabilistic …

Content truncated to 500 words