Exploring Content Models for Multi-Document Summarization

Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the ACL, pages 362–370, Boulder, Colorado, June 2009. c⃝2009 Association for Computational Linguistics Exploring Content Models for Multi-Document Summarization Aria Haghighi UC Berkeley, CS Division aria42@cs.berkeley.edu Lucy Vanderwende Microsoft Research Lucy.Vanderwende@microsoft.com Abstract We present an exploration of generative prob- abilistic models for multi-document summa- rization. Beginning with a simple word fre- quency based model (Nenkova and Vander- wende, 2005), we construct a sequence of models each injecting more structure into the representation of document set content and ex- hibiting ROUGE gains along the way. Our final model, HIERSUM, utilizes a hierarchi- cal LDA-style model (Blei et al., 2004) to represent content specificity as a hierarchy of topic vocabulary distributions. At the task of producing generic DUC-style summaries, HIERSUM yields state-of-the-art ROUGE per- formance and in pairwise user evaluation strongly outperforms Toutanova et al. (2007)’s state-of-the-art discriminative system. We also explore HIERSUM’s capacity to produce multiple ‘topical summaries’ in order to facil- itate content discovery and navigation. 1 Introduction Over the past several years, there has been much in- terest in the task of multi-document summarization. In the common Document Understanding Confer- ence (DUC) formulation of the task, a system takes as input a document set as well as a short descrip- tion of desired summary focus and outputs a word length limited summary.1 To avoid the problem of generating cogent sentences, many systems opt for an extractive approach, selecting sentences from the document set which best reflect its core content.2 1In this work, we ignore the summary focus. Here, the word topic will refer to elements of our statistical model rather than summary focus. 2Note that sentence extraction does not solve the problem of selecting and ordering summary sentences to form a coherent There are several approaches to modeling docu- ment content: simple word frequency-based meth- ods (Luhn, 1958; Nenkova and Vanderwende, 2005), graph-based approaches (Radev, 2004; Wan and Yang, 2006), as well as more linguistically mo- tivated techniques (Mckeown et al., 1999; Leskovec et al., 2005; Harabagiu et al., 2007). Another strand of work (Barzilay and Lee, 2004; Daum´e III and Marcu, 2006; Eisenstein and Barzilay, 2008), has explored the use of structured probabilistic topic models to represent document content. However, lit- tle has been done to directly compare the benefit of complex content models to simpler surface ones for generic multi-document summarization. In this work we examine a series of content models for multi-document summarization and ar- gue that LDA-style probabilistic topic models (Blei et al., 2003) can offer state-of-the-art summariza- tion quality as measured by automatic metrics (see section 5.1) and manual user evaluation (see sec- tion 5.2). We also contend that they provide con- venient building blocks for adding more structure to a summarization model. In particular, we uti- lize a variation of the hierarchical LDA topic model (Blei et al., 2004) to discover multiple specific ‘sub- topics’ within a document set. The resulting model, HIERSUM (see section 3.4), can produce general summaries …

Content truncated to 500 words