Overcoming the Lack of Parallel Data in Sentence Compression

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1481–1491, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Overcoming the Lack of Parallel Data in Sentence Compression Katja Filippova and Yasemin Altun Google Brandschenkestr. 110 Z¨urich, 8004 Switzerland katjaf|altun@google.com Abstract A major challenge in supervised sentence compression is making use of rich feature rep- resentations because of very scarce parallel data. We address this problem and present a method to automatically build a compres- sion corpus with hundreds of thousands of instances on which deletion-based algorithms can be trained. In our corpus, the syntactic trees of the compressions are subtrees of their uncompressed counterparts, and hence super- vised systems which require a structural align- ment between the input and output can be suc- cessfully trained. We also extend an exist- ing unsupervised compression method with a learning module. The new system uses struc- tured prediction to learn from lexical, syntac- tic and other features. An evaluation with hu- man raters shows that the presented data har- vesting method indeed produces a parallel cor- pus of high quality. Also, the supervised sys- tem trained on this corpus gets high scores both from human raters and in an automatic evaluation setting, significantly outperforming a strong baseline. 1 Introduction and related work Sentence compression is a paraphrasing task where the goal is to generate sentences shorter than given while preserving the essential content. A robust compression system would be useful for mobile de- vices as well as a module in an extractive sum- marization system (Mani, 2001). Although a com- pression may differ lexically and structurally from the source sentence, to date most systems are ex- tractive and proceed by deleting words from the input (Knight & Marcu, 2000; Dorr et al., 2003; Turner & Charniak, 2005; Clarke & Lapata, 2008; Berg-Kirkpatrick et al., 2011, inter alia). To de- cide which words, dependencies or phrases can be dropped, (i) rule-based approaches (Grefenstette, 1998; Jing & McKeown, 2000; Dorr et al., 2003; Zajic et al., 2007), (ii) supervised models trained on parallel data (Knight & Marcu, 2000; Turner & Charniak, 2005; McDonald, 2006; Gillick & Favre, 2009; Galanis & Androutsopoulos, 2010, inter alia) and (iii) unsupervised methods which make use of statistics collected from non-parallel data (Hori & Furui, 2004; Zajic et al., 2007; Clarke & Lapata, 2008; Filippova & Strube, 2008) have been investi- gated. Since it is infeasible to manually devise a set of accurate deletion rules with high coverage, recent research has been devoted to developing statistical methods and possibly augmenting them with a few linguistic rules to improve output readability (Clarke & Lapata, 2008; Nomoto, 2009). Supervised models. A major problem for super- vised deletion-based systems is very limited amount of parallel data. Many approaches make use of a small portion of the Ziff-Davis corpus which has about 1K sentence-compression pairs1. Other main sources of training data are the two manually crafted compression corpora from the University of Edin- burgh (“written” and “spoken”, …

Content truncated to 500 words