Jointly Learning to Extract and Compress

Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 481–490, Portland, Oregon, June 19-24, 2011. c⃝2011 Association for Computational Linguistics Jointly Learning to Extract and Compress Taylor Berg-Kirkpatrick Dan Gillick Dan Klein Computer Science Division University of California at Berkeley {tberg, dgillick, klein}@cs.berkeley.edu Abstract We learn a joint model of sentence extraction and compression for multi-document summa- rization. Our model scores candidate sum- maries according to a combined linear model whose features factor over (1) the n-gram types in the summary and (2) the compres- sions used. We train the model using a margin- based objective whose loss captures end sum- mary quality. Because of the exponentially large set of candidate summaries, we use a cutting-plane algorithm to incrementally de- tect and add active constraints efficiently. In- ference in our model can be cast as an ILP and thereby solved in reasonable time; we also present a fast approximation scheme which achieves similar performance. Our jointly extracted and compressed summaries outper- form both unlearned baselines and our learned extraction-only system on both ROUGE and Pyramid, without a drop in judged linguis- tic quality. We achieve the highest published ROUGE results to date on the TAC 2008 data set. 1 Introduction Applications of machine learning to automatic sum- marization have met with limited success, and, as a result, many top-performing systems remain largely ad-hoc. One reason learning may have provided lim- ited gains is that typical models do not learn to opti- mize end summary quality directly, but rather learn intermediate quantities in isolation. For example, many models learn to score each input sentence in- dependently (Teufel and Moens, 1997; Shen et al., 2007; Schilder and Kondadadi, 2008), and then as- semble extractive summaries from the top-ranked sentences in a way not incorporated into the learn- ing process. This extraction is often done in the presence of a heuristic that limits redundancy. As another example, Yih et al. (2007) learn predictors of individual words’ appearance in the references, but in isolation from the sentence selection proce- dure. Exceptions are Li et al. (2009) who take a max-margin approach to learning sentence values jointly, but still have ad hoc constraints to handle redundancy. One main contribution of the current paper is the direct optimization of summary quality in a single model; we find that our learned systems substantially outperform unlearned counterparts on both automatic and manual metrics. While pure extraction is certainly simple and does guarantee some minimal readability, Lin (2003) showed that sentence compression (Knight and Marcu, 2001; McDonald, 2006; Clarke and Lap- ata, 2008) has the potential to improve the resulting summaries. However, attempts to incorporate com- pression into a summarization system have largely failed to realize large gains. For example, Zajic et al (2006) use a pipeline approach, pre-processing to yield additional candidates for extraction by ap- plying heuristic sentence compressions, but their system does not outperform state-of-the-art purely extractive systems. Similarly, Gillick and Favre (2009), though not learning weights, do a limited form of …

Content truncated to 500 words