Models for Sentence Compression: A Comparison across Domains, Training Requirements and Evaluation Measures

Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 377–384, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Models for Sentence Compression: A Comparison across Domains, Training Requirements and Evaluation Measures James Clarke and Mirella Lapata School of Informatics, University of Edinburgh 2 Bucclecuch Place, Edinburgh EH8 9LW, UK jclarke@ed.ac.uk, mlap@inf.ed.ac.uk Abstract Sentence compression is the task of pro- ducing a summary at the sentence level. This paper focuses on three aspects of this task which have not received de- tailed treatment in the literature: train- ing requirements, scalability, and auto- matic evaluation. We provide a novel com- parison between a supervised constituent- based and an weakly supervised word- based compression algorithm and exam- ine how these models port to different do- mains (written vs. spoken text). To achieve this, a human-authored compression cor- pus has been created and our study high- lights potential problems with the auto- matically gathered compression corpora currently used. Finally, we assess whether automatic evaluation measures can be used to determine compression quality. 1 Introduction Automatic sentence compression has recently at- tracted much attention, in part because of its affin- ity with summarisation. The task can be viewed as producing a summary of a single sentence that retains the most important information while re- maining grammatically correct. An ideal compres- sion algorithm will involve complex text rewriting operations such as word reordering, paraphrasing, substitution, deletion, and insertion. In default of a more sophisticated compression algorithm, cur- rent approaches have simplified the problem to a single rewriting operation, namely word deletion. More formally, given an input sentence of words W = w1,w2,...,wn, a compression is formed by dropping any subset of these words. Viewing the task as word removal reduces the number of pos- sible compressions to 2n; naturally, many of these compressions will not be reasonable or grammati- cal (Knight and Marcu 2002). Sentence compression could be usefully em- ployed in wide range of applications. For exam- ple, to automatically generate subtitles for televi- sion programs; the transcripts cannot usually be used verbatim due to the rate of speech being too high (Vandeghinste and Pan 2004). Other applica- tions include compressing text to be displayed on small screens (Corston-Oliver 2001) such as mo- bile phones or PDAs, and producing audio scan- ning devices for the blind (Grefenstette 1998). Algorithms for sentence compression fall into two broad classes depending on their training re- quirements. Many algorithms exploit parallel cor- pora (Jing 2000; Knight and Marcu 2002; Riezler et al. 2003; Nguyen et al. 2004a; Turner and Char- niak 2005; McDonald 2006) to learn the corre- spondences between long and short sentences in a supervised manner, typically using a rich feature space induced from parse trees. The learnt rules effectively describe which constituents should be deleted in a given context. Approaches that do not employ parallel corpora require minimal or no supervision. They operationalise compression in terms of word deletion without learning spe- cific rules and can therefore rely …

Content truncated to 500 words