Supervised and Unsupervised Learning for Sentence Compression

Proceedings of the 43rd Annual Meeting of the ACL, pages 290–297, Ann Arbor, June 2005. c⃝2005 Association for Computational Linguistics Supervised and Unsupervised Learning for Sentence Compression Jenine Turner and Eugene Charniak Department of Computer Science Brown Laboratory for Linguistic Information Processing (BLLIP) Brown University Providence, RI 02912 {jenine|ec}@cs.brown.edu Abstract In Statistics-Based Summarization - Step One: Sentence Compression, Knight and Marcu (Knight and Marcu, 2000) (K&M) present a noisy-channel model for sen- tence compression. The main difficulty in using this method is the lack of data; Knight and Marcu use a corpus of 1035 training sentences. More data is not easily available, so in addition to improving the original K&M noisy-channel model, we create unsupervised and semi-supervised models of the task. Finally, we point out problems with modeling the task in this way. They suggest areas for future re- search. 1 Introduction Summarization in general, and sentence compres- sion in particular, are popular topics. Knight and Marcu (henceforth K&M) introduce the task of statistical sentence compression in Statistics-Based Summarization - Step One: Sentence Compression (Knight and Marcu, 2000). The appeal of this prob- lem is that it produces summarizations on a small scale. It simplifies general compression problems, such as text-to-abstract conversion, by eliminating the need for coherency between sentences. The model is further simplified by being constrained to word deletion: no rearranging of words takes place. Others have performed the sentence compres- sion task using syntactic approaches to this problem (Mani et al., 1999) (Zajic et al., 2004), but we fo- cus exclusively on the K&M formulation. Though the problem is simpler, it is still pertinent to cur- rent needs; generation of captions for television and audio scanning services for the blind (Grefenstette, 1998), as well as compressing chosen sentences for headline generation (Angheluta et al., 2004) are ex- amples of uses for sentence compression. In addi- tion to simplifying the task, K&M’s noisy-channel formulation is also appealing. In the following sections, we discuss the K&M noisy-channel model. We then present our cleaned up, and slightly improved noisy-channel model. We also develop unsupervised and semi-supervised (our term for a combination of supervised and unsuper- vised) methods of sentence compression with inspi- ration from the K&M model, and create additional constraints to improve the compressions. We con- clude with the problems inherent in both models. 2 The Noisy-Channel Model 2.1 The K&M Model The K&M probabilistic model, adapted from ma- chine translation to this task, is the noisy-channel model. In machine translation, one imagines that a string was originally in English, but that someone adds some noise to make it a foreign string. Analo- gously, in the sentence compression model, the short string is the original sentence and someone adds noise, resulting in the longer sentence. Using this framework, the end goal is, given a long sentence l, to determine the short sentence s that maximizes 290 P(s | l). By Bayes Rule, P(s | l) = P(l | s)P(s) P(l) (1) The probability of the long sentence, P(l) can …

Content truncated to 500 words