Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 1998–2008, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints Greg Durrett Computer Science Division UC Berkeley gdurrett@cs.berkeley.edu Taylor Berg-Kirkpatrick School of Computer Science Carnegie Mellon University tberg@cs.cmu.edu Dan Klein Computer Science Division UC Berkeley klein@cs.berkeley.edu Abstract We present a discriminative model for single-document summarization that integrally combines compression and anaphoricity constraints. Our model selects textual units to include in the summary based on a rich set of sparse features whose weights are learned on a large corpus. We allow for the deletion of content within a sentence when that deletion is licensed by compression rules; in our framework, these are implemented as dependencies between subsentential units of text. Anaphoricity constraints then improve cross-sentence coherence by guaranteeing that, for each pronoun included in the summary, the pronoun’s antecedent is included as well or the pronoun is rewritten as a full mention. When trained end-to-end, our final sys- tem1 outperforms prior work on both ROUGE as well as on human judgments of linguistic quality. 1 Introduction While multi-document summarization is well- studied in the NLP literature (Carbonell and Gold- stein, 1998; Gillick and Favre, 2009; Lin and Bilmes, 2011; Nenkova and McKeown, 2011), single-document summarization (McKeown et al., 1995; Marcu, 1998; Mani, 2001; Hirao et al., 2013) has received less attention in recent years and is generally viewed as more difficult. Con- tent selection is tricky without redundancy across multiple input documents as a guide and sim- ple positional information is often hard to beat (Penn and Zhu, 2008). In this work, we tackle the single-document problem by training an ex- pressive summarization model on a large nat- 1Available at http://nlp.cs.berkeley.edu urally occurring corpus—the New York Times Annotated Corpus (Sandhaus, 2008) which con- tains around 100,000 news articles with abstrac- tive summaries—learning to select important con- tent with lexical features. This corpus has been explored in related contexts (Dunietz and Gillick, 2014; Hong and Nenkova, 2014), but to our knowledge it has not been directly used for single- document summarization. To increase the expressive capacity of our model we allow more aggressive compression of individual sentences by combining two different formalisms—one syntactic and the other discur- sive. Additionally, we incorporate a model of anaphora resolution and give our system the abil- ity rewrite pronominal mentions, further increas- ing expressivity. In order to guide the model, we incorporate (1) constraints from coreference en- suring that critical pronoun references are clear in the final summary and (2) constraints from syntac- tic and discourse parsers ensuring that sentence re- alizations are well-formed. Despite the complex- ity of these additional constraints, we demonstrate an efficient inference procedure using an ILP- based approach. By training our full system end- to-end on a large-scale dataset, we are able to learn a high-capacity structured model of the summa- rization process, contrasting with past approaches to the single-document task which have typically been heuristic in nature (Daum´e …

Content truncated to 500 words