A New Sentence Reduction based on Decision Tree Model

A New Sentence Reduction based on Decisions tree model Nguyen Minh Le Graduate School of Information Science, JAIST ISHIKAWA 923-1292, JAPAN nguyenml@jaist.ac.jp Susumu Horiguchi Graduate School of Information Science, JAIST ISHIKAWA 923-1292, JAPAN hori@jaist.ac.jp Abstract This paper addresses a novel sentence reduction algorithm base on decision tree model where semantic information is used to enhance the accuracy of sentence reduction. The proposed algorithm is able to deal with the changeable order problem in sentence reduction. Experimental show a better result when comparing with the original methods. 1� Introduction Many researches in automatic text summarization were focused on extraction or identifying the important clauses and sentences, paragraphs in texts (Math and Maybury, 1999). Meanwhile, humans used to produce summaries by creating new sentences that are grammatical, that cohere with one another, and capture the most salient parts of information in the original document. Sentence reduction is the problem of removing some redundant words or some phrases from the original sentence by creating a new sentence, in which the gist meaning of the original sentence was =changed . Methods of sentence reduction have been applied in many applications. Grefenstette (Grefenstette,S, 1998) proposed removing phrases in sentences to produce a telegraphic text that can be used to provide audio scanning services for the blind. Dolan (Donlan,W.B, 1999) proposed removing clauses in sentences before indexing document for information retrieval. Those methods removed phrases based on their syntactic categories without relying on the context of words, phrases and sentences around. Therefore, those methods are unsuitable for text summarization task. Sentence reduction for text summarization is pointed out by Mani and Maybury (Mani and Maybury,1999). The authors present a process of writing reduced sentences by reversing the original sentence with a set of revised rules. Jing (Jing,H, 2000) also studied a method to remove extraneous phrases from sentences by using multiple source of knowledge to decide which phrase can be removed. The multiple sources include syntactic knowledge, context information and statistic computed from a corpus that consists of examples written by human professional. Their method prevented removing some phrases that were relative to its context around and produced a grammatical sentence, and applied to the cut and paste summarization strategy. Recently, Knight and Marcus (Knight and Marcu,D, 2002) demonstrated two methods for sentence compression problem based on corpus. They devised both noisy-channel and decision tree approach to the problem. The decision tree approach has been applied in parsing sentence (Magerman,D,1995) (Hermijakob,U and Mooney,R, 1997) and defining the rhetorical of text documents (Marcu,D,1999) and achieved a good results in sentence compression as described in (Knight and Marcu,D, 2002). In almost previous methods, the order of reduced sentences is the same with the original sentence. Meanwhile, in summrizing document, human may perform a changeable order to ensure the summary document is smooth and coherence. This fact requires a new sentence reduction with the order of reduced sentence is different from the orignal. In addition to using sentence reduction for text summarization, the information of syntactic is not enough. …

Content truncated to 500 words