A Novel Feature-based Bayesian Model for Query Focused Multi-document Summarization

Transactions of the Association for Computational Linguistics, 1 (2013) 89–98. Action Editor: Noah Smith. Submitted 12/2012; Published 5/2013. c⃝2013 Association for Computational Linguistics. A Novel Feature-based Bayesian Model for Query Focused Multi-document Summarization Jiwei Li School of Computer Science Carnegie Mellon University bdlijiwei@gmail.com Sujian Li Laboratory of Computational Linguistics Peking University lisujian@pku.edu.cn Abstract Supervised learning methods and LDA based topic model have been successfully applied in the field of multi-document summarization. In this paper, we propose a novel supervised approach that can in- corporate rich sentence features into Bayesian topic models in a principled way, thus taking advantages of both topic model and feature based supervised learn- ing methods. Experimental results on DUC2007, TAC2008 and TAC2009 demonstrate the effective- ness of our approach. 1 Introduction Query-focused multi-document summarization (Nenkova et al., 2006; Wan et al., 2007; Ouyang et al., 2010) can facilitate users to grasp the main idea of documents. In query-focused summarization, a specific topic description, such as a query, which expresses the most important topic information is proposed before the document collection, and a summary would be generated according to the given topic. Supervised models have been widely used in sum- marization (Li, et al., 2009, Shen et al., 2007, Ouyang et al., 2010). Supervised models usually re- gard summarization as a classification or regression problem and use various sentence features to build a classifier based on labeled negative or positive sam- ples. However, existing supervised approaches sel- dom exploit the intrinsic structure among sentences. This disadvantage usually gives rise to serious prob- lems such as unbalance and low recall in summaries. Recently, LDA-based (Blei et al., 2003) Bayesian topic models have widely been applied in multi- document summarization in that Bayesian ap- proaches can offer clear and rigorous probabilis- tic interpretations for summaries(Daume and Marcu, 2006; Haghighi and Vanderwende, 2009; Jin et al., 2010; Mason and Charniak, 2011; Delort and Alfon- seca, 2012). Exiting Bayesian approaches label sen- tences or words with topics and sentences which are closely related with query or can highly generalize documents are selected into summaries. However, LDA topic model suffers from the intrinsic disad- vantages that it only uses word frequency for topic modeling and can not use useful text features such as position, word order etc (Zhu and Xing, 2010). For example, the first sentence in a document may be more important for summary since it is more likely to give a global generalization about the document. It is hard for LDA model to consider such informa- tion, making useful information lost. It naturally comes to our minds that we can im- prove summarization performance by making full use of both useful text features and the latent seman- tic structures from by LDA topic model. One related work is from Celikyilmaz and Hakkani-Tur (2010). They built a hierarchical topic model called Hybh- sum based on LDA for topic discovery and assumed this model can produce appropriate scores for sen- tence evaluation. Then the scores are used for tun- ing the weights …

Content truncated to 500 words