An Unsupervised Multi-Document Summarization Framework Based on Neural Document Model

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1514–1523, Osaka, Japan, December 11-17 2016. An Unsupervised Multi-Document Summarization Framework Based on Neural Document Model Shulei Ma Zhi-Hong Deng∗ Key Laboratory of Machine Perception (Ministry of Education) School of Electronics Engineering and Computer Science Peking University, Beijing 100871, China mashulei@pku.edu.cn, zhdeng@cis,pku.edu.cn incomparable-lun@pku.edu.cn Yunlun Yang Abstract In the age of information exploding, multi-document summarization is attracting particular atten- tion for the ability to help people get the main ideas in a short time. Traditional extractive meth- ods simply treat the document set as a group of sentences while ignoring the global semantics of the documents. Meanwhile, neural document model is effective on representing the semantic content of documents in low-dimensional vectors. In this paper, we propose a document-level reconstruction framework named DocRebuild, which reconstructs the documents with summary sentences through a neural document model and selects summary sentences to minimize the re- construction error. We also apply two strategies, sentence filtering and beamsearch, to improve the performance of our method. Experimental results on the benchmark datasets DUC 2006 and DUC 2007 show that DocRebuild is effective and outperforms the state-of-the-art unsupervised algorithms. 1 Introduction Multi-document summarization aims at capturing the important information of a set of documents related to the same topic and presenting it in a brief, representative, and pertinent summary. Most existing re- searches focus on extraction-based methods, in which sentences are selected from the original document set. Typically, two kinds of unsupervised models are used for sentence selection. One is based on sentence ranking, which uses methods such as clustering (Lin and Hovy, 2002; Radev et al., 2004), PageRank (Erkan and Radev, 2004; Mihalcea and Tarau, 2005) and topic modeling (Harabagiu and Lacatusu, 2005; Wang et al., 2008), to rank the sentences. Considering that top-ranked sentences tend to convey much redundant information, additional strategies are usually applied to reduce redundancy when selecting sentences. This kind of methods usually need to weigh between relevance and redundancy, which may be hard to balance. The other is based on sparse reconstruction (He et al., 2012; Liu et al., 2015; Yao et al., 2015), which selects a sparse subset of the sentences that can linearly reconstruct all the sentences in the original document set. This kind of methods has a good motivation but also weaknesses in their hypothesises. First, reconstructing single sentences may lose the global information of documents; Second, there are more reasonable ways than linear combination in reconstruction. Commonly, in the above methods the document set is treated as a set of sentences and all the operations are carried out on the sentence set, losing the global information of documents. Meanwhile, neural document model (Le and Mikolov, 2014; Li et al., 2015; Lin et al., 2015) is an emerging technique which has made significant progress in capturing semantic information of documents by projecting the text into the low-dimensional continuous distributed representation. It has been applied to natural language processing tasks such as sentiment classification (Tang …

Content truncated to 500 words