Extractive Summarization by Maximizing Semantic Volume

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1961–1966, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Extractive Summarization by Maximizing Semantic Volume Dani Yogatama School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213, USA dyogatama@cs.cmu.edu Fei Liu Electrical Engineering & Computer Science University of Central Florida Orlando, FL 32816, USA feiliu@cs.ucf.edu Noah A. Smith Computer Science & Engineering University of Washington Seattle, WA 98195, USA nasmith@cs.washington.edu Abstract The most successful approaches to extrac- tive text summarization seek to maximize bigram coverage subject to a budget con- straint. In this work, we propose instead to maximize semantic volume. We em- bed each sentence in a semantic space and construct a summary by choosing a sub- set of sentences whose convex hull max- imizes volume in that space. We provide a greedy algorithm based on the Gram- Schmidt process to efficiently perform volume maximization. Our method out- performs the state-of-the-art summariza- tion approaches on benchmark datasets. 1 Introduction In artificial intelligence, changes in representation sometimes suggest new algorithms. For example, increased attention to distributed meaning repre- sentations suggests that existing combinatorial al- gorithms for NLP might be supplanted by alterna- tives designed specifically for embeddings. In this work, we consider summarization. Classical approaches to extractive summariza- tion represent each sentence as a bag of terms (typically bigrams) and seek a subset of sentences from the input document(s) that either (a) trade off between high relevance and low redundancy (Car- bonell and Goldstein, 1998; McDonald, 2007), or (b) maximize bigram coverage (Yih et al., 2007; Gillick et al., 2008). The sentence representa- tion is fundamentally discrete, and a range of greedy (Carbonell and Goldstein, 1998), approx- imate (Almeida and Martins, 2013), and exact op- timization algorithms (McDonald, 2007; Martins and Smith, 2009; Berg-Kirkpatrick et al., 2011) have been proposed. Recent studies have explored continuous sen- tence representations, including the paragraph vector (Le and Mikolov, 2014), a convolutional neural network architecture (Kalchbrenner et al., 2014), and a dictionary learning approach (Jenat- ton et al., 2011). If sentences are represented as low-dimensional embeddings in a distributed se- mantic space, then we begin to imagine a geomet- ric relationship between a summary and a doc- ument. We propose that the volume of a sum- mary (i.e., the semantic subspace spanned by the selected sentences) should ideally be large. We therefore formalize a new objective function for summarization based on semantic volume (§2), and we provide a fast greedy algorithm that can be used to maximize it (§3). We show that our method outperforms competing extractive base- lines under similar experimental conditions on benchmark summarization datasets (§4). 2 Extractive Summarization Models Assume we are given a set of N sentences: D = {s1, s2, . . . , sN} from one or many documents, and the goal is to produce a summary by choos- ing a subset S of M sentences, where S ⊆ D and M ≤ N, and the length of the summary is less than …

Content truncated to 500 words