A Noisy-Channel Model for Document Compression

A Noisy-Channel Model for Document Compression Hal Daum´e III and Daniel Marcu Information Sciences Institute University of Southern California 4676 Admiralty Way, Suite 1001 Marina del Rey, CA 90292 � hdaume,marcu � @isi.edu Abstract We present a document compression sys- tem that uses a hierarchical noisy-channel model of text production. Our compres- sion system first automatically derives the syntactic structure of each sentence and the overall discourse structure of the text given as input. The system then uses a sta- tistical hierarchical model of text produc- tion in order to drop non-important syn- tactic and discourse constituents so as to generate coherent, grammatical document compressions of arbitrary length. The sys- tem outperforms both a baseline and a sentence-based compression system that operates by simplifying sequentially all sentences in a text. Our results support the claim that discourse knowledge plays an important role in document summariza- tion. 1 Introduction Single document summarization systems proposed to date fall within one of the following three classes: Extractive summarizers simply select and present to the user the most important sentences in a text — see (Mani and Maybury, 1999; Marcu, 2000; Mani, 2001) for comprehensive overviews of the methods and algorithms used to accomplish this. Headline generators are noisy-channel probabilis- tic systems that are trained on large corpora of � Headline, Text � pairs (Banko et al., 2000; Berger and Mittal, 2000). These systems pro- duce short sequences of words that are indica- tive of the content of the text given as input. Sentence simplification systems (Chandrasekar et al., 1996; Mahesh, 1997; Carroll et al., 1998; Grefenstette, 1998; Jing, 2000; Knight and Marcu, 2000) are capable of compressing long sentences by deleting unimportant words and phrases. Extraction-based summarizers often produce out- puts that contain non-important sentence fragments. For example, the hypothetical extractive summary of Text (1), which is shown in Table 1, can be com- pacted further by deleting the clause “which is al- ready almost enough to win”. Headline-based sum- maries, such as that shown in Table 1, are usually indicative of a text’s content but not informative, grammatical, or coherent. By repeatedly applying a sentence-simplification algorithm one sentence at a time, one can compress a text; yet, the outputs gen- erated in this way are likely to be incoherent and to contain unimportant information. When summa- rizing text, some sentences should be dropped alto- gether. Ideally, we would like to build systems that have the strengths of all these three classes of approaches. The “Document Compression” entry in Table 1 shows a grammatical, coherent summary of Text (1), which was generated by a hypothetical document compression system that preserves the most impor- tant information in a text while deleting sentences, phrases, and words that are subsidiary to the main message of the text. Obviously, generating coher- ent, grammatical summaries such as that produced by the hypothetical document compression system in Table 1 is not trivial because of many conflicting Computational Linguistics (ACL), Philadelphia, July 2002, pp. 449-456. Proceedings of the 40th Annual …

Content truncated to 500 words