UPER: Boosting Multi-Document Summarization with an Unsupervised Prompt-based Extractor

Proceedings of the 29th International Conference on Computational Linguistics, pages 6315–6326 October 12–17, 2022. 6315 UPER: Boosting Multi-Document Summarization with an Unsupervised Prompt-based Extractor Shangqing Tu1,2, Jifan Yu1,2, Fangwei Zhu1,2, Juanzi Li1,2, Lei Hou1,2∗and Jian-Yun Nie3 1Dept. of Computer Sci.&Tech., BNRist, Tsinghua University, Beijing 100084, China 2KIRC, Institute for Artificial Intelligence, Tsinghua University 3Department of Computer Science and Operations Research, University of Montreal, Canada {tsq22,yujf21,zfw19}@mails.tsinghua.edu.cn {lijuanzi,houlei}@tsinghua.edu.cn nie@iro.umontreal.ca Abstract Multi-Document Summarization (MDS) com- monly employs the 2-stage extract-then- abstract paradigm, which first extracts a rel- atively short meta-document, then feeds it into the deep neural networks to generate an ab- stract. Previous work usually takes the ROUGE score as the label for training a scoring model to evaluate source documents. However, the trained scoring model is prone to under-fitting for low-resource settings, as it relies on the training data. To extract documents effectively, we construct prompting templates that invoke the underlying knowledge in Pre-trained Lan- guage Model (PLM) to calculate the document and keyword’s perplexity, which can assess the document’s semantic salience. Our unsuper- vised approach can be applied as a plug-in to boost other metrics for evaluating a document’s salience, thus improving the subsequent ab- stract generation. We get positive results on 2 MDS datasets, 2 data settings, and 2 abstrac- tive backbone models, showing our method’s effectiveness. Our code is available at https: //github.com/THU-KEG/UPER 1 Introduction Multi-Document Summarization (MDS) aims to generate a summary from multiple source arti- cles (McKeown and Radev, 1995). The input text in MDS can be overlong and therefore contain much noisy information (Liu et al., 2021c). The goal of MDS is to reduce the long input and extract salient information (Bing et al., 2015; Fan et al., 2019; Song et al., 2022). Some previous works (Zaheer et al., 2020) utilise sparse attention to handle the long input problem in MDS. Many others tackle this problem by an extract-then-abstract paradigm (Liu and La- pata, 2019), which first extracts the salient informa- tion in source documents to form a meta-document with a preset length then generates the summary. ∗Corresponding author. The first stage reduces the input length for the sec- ond stage to cut the abstractive model’s memory cost. The extractive stage has two main technical lines: (1) the statistical method uses the token-level similarity between keywords and source documents to retrieve relevant documents (Liu et al., 2018). (2) the regressive method trains a scoring model to pre- dict the document’s ROUGE (Lin, 2004) score with the reference summary (Liu and Lapata, 2019; Mao et al., 2021; Zhong et al., 2021; Zhang et al., 2021). However, these methods for the extractive stage lead to several problems: (1) the statistical meth- ods like tf-idf (Ramos et al., 2003) relies on strict matching of keywords, ignoring those documents with relevant semantic context. (2) the regressive methods aim to fit some predefined metrics, e.g., ROUGE, but the fitting result seriously depends on the training data, leading to over-fitting or under- fitting. Besides, the predefined metrics may not adequately measure …

Content truncated to 500 words