Fast and Robust Compressive Summarization with Dual Decomposition and Multi-Task Learning

Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 196–206, Sofia, Bulgaria, August 4-9 2013. c⃝2013 Association for Computational Linguistics Fast and Robust Compressive Summarization with Dual Decomposition and Multi-Task Learning Miguel B. Almeida∗† Andr´e F. T. Martins∗† ∗Priberam Labs, Alameda D. Afonso Henriques, 41, 2o, 1000-123 Lisboa, Portugal †Instituto de Telecomunicac¸˜oes, Instituto Superior T´ecnico, 1049-001 Lisboa, Portugal {mba,atm}@priberam.pt Abstract We present a dual decomposition frame- work for multi-document summarization, using a model that jointly extracts and compresses sentences. Compared with previous work based on integer linear pro- gramming, our approach does not require external solvers, is significantly faster, and is modular in the three qualities a sum- mary should have: conciseness, informa- tiveness, and grammaticality. In addition, we propose a multi-task learning frame- work to take advantage of existing data for extractive summarization and sentence compression. Experiments in the TAC- 2008 dataset yield the highest published ROUGE scores to date, with runtimes that rival those of extractive summarizers. 1 Introduction Automatic text summarization is a seminal prob- lem in information retrieval and natural language processing (Luhn, 1958; Baxendale, 1958; Ed- mundson, 1969). Today, with the overwhelming amount of information available on the Web, the demand for fast, robust, and scalable summariza- tion systems is stronger than ever. Up to now, extractive systems have been the most popular in multi-document summarization. These systems produce a summary by extracting a representative set of sentences from the origi- nal documents (Kupiec et al., 1995; Carbonell and Goldstein, 1998; Radev et al., 2000; Gillick et al., 2008). This approach has obvious advantages: it reduces the search space by letting decisions be made for each sentence as a whole (avoiding fine- grained text generation), and it ensures a grammat- ical summary, assuming the original sentences are well-formed. The typical trade-offs in these mod- els (maximizing relevance, and penalizing redun- dancy) lead to submodular optimization problems (Lin and Bilmes, 2010), which are NP-hard but ap- proximable through greedy algorithms; learning is possible with standard structured prediction algo- rithms (Sipos et al., 2012; Lin and Bilmes, 2012). Probabilistic models have also been proposed to capture the problem structure, such as determinan- tal point processes (Gillenwater et al., 2012). However, extractive systems are rather limited in the summaries they can produce. Long, partly relevant sentences tend not to appear in the sum- mary, or to block the inclusion of other sen- tences. This has motivated research in compres- sive summarization (Lin, 2003; Zajic et al., 2006; Daum´e, 2006), where summaries are formed by compressed sentences (Knight and Marcu, 2000), not necessarily extracts. While promising results have been achieved by models that simultaneously extract and compress (Martins and Smith, 2009; Woodsend and Lapata, 2010; Berg-Kirkpatrick et al., 2011), there are still obstacles that need to be surmounted for these systems to enjoy wide adoption. All approaches above are based on in- teger linear programming (ILP), suffering from slow runtimes, when compared to extractive sys- tems. For example, Woodsend and Lapata (2012) report 55 seconds …

Content truncated to 500 words