Non-Parametric Memory Guidance for Multi-Document Summarization

Proceedings of Recent Advances in Natural Language Processing, pages 153–158 Varna, Sep 4–6, 2023 https://doi.org/10.26615/978-954-452-092-2_017 153 Non-Parametric Memory Guidance for Multi-Document Summarization Florian Baud LIRIS, Villeurbanne Visiativ florian.baud@visiativ.com Alex Aussem LIRIS, Villeurbanne alexandre.aussem@liris.cnrs.fr Abstract Multi-document summarization (MDS) is a difficult task in Natural Language Processing, aiming to summarize information from sev- eral documents. However, the source docu- ments are often insufficient to obtain a qualita- tive summary. We propose a retriever-guided model combined with non-parametric memory for summary generation. This model retrieves relevant candidates from a database and then generates the summary considering the candi- dates with a copy mechanism and the source documents. The retriever is implemented with Approximate Nearest Neighbor Search (ANN) to search large databases. Our method is eval- uated on the MultiXScience dataset which in- cludes scientific articles. Finally, we discuss our results and possible directions for future work. 1 Introduction Multi-document summarization is performed using two methods: extractive (Wang et al., 2020; Liu et al., 2021) or abstractive (Jin et al., 2020; Xiao et al., 2022). So-called extractive methods rank sen- tences from source documents that best summarize them. These methods reuse important information well to construct a good summary but they lack coherence between sentences. To overcome this issue, abstractive methods are studied to imitate human writing behavior. They show great perfor- mance in human writing style but they often miss key information. To make abstractive models aware of essential information, (Dou et al., 2021) guides their model with additional information like a set of keywords, graph triples, highlighted sentences of source doc- uments, or retrieved similar summaries. Their method, which uses every guidance previously mentioned, improves summary quality and control- lability compared with unguided models. However, guidances require specific training data, especially for keywords, graph triples, and highlighted sen- tences. Our proposal is that by guiding with pre-existing summaries, the model can draw inspiration from the summary as a whole. But also be able to ex- tract keywords and phrases using a copy mecha- nism. Consequently, this work focuses on guidance by similar summaries extracted from a knowledge base using a similarity metric between source doc- uments and pre-existing summaries. The model, inspired by RAG (Lewis et al., 2020), is fully dif- ferentiable. In addition, the model generator uses a copy mechanism on the candidates returned from the knowledge base, inspired by (Cai et al., 2021). The findings of these two studies motivated the de- velopment of our model for the multi-document text summarization task. We demonstrate the potential of our method on MultiXScience (Lu et al., 2020). This dataset gath- ers scientific articles where we have to generate the ”related work” part with the ”abstract” of the source article and the ”abstracts” of the citations. In the case of scientific articles, we believe that the source documents are insufficient to generate the ”related work” part because external knowledge is necessary to write such a paragraph. In this work, we investigate a sequence-to- sequence model guided by a memory retriever of similar summaries. …

Content truncated to 500 words