SummVD : An efficient approach for unsupervised topic-based text summarization

Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 501–511 November 20–23, 2022. ©2022 Association for Computational Linguistics 501 SummVD : An efficient approach for unsupervised topic-based text summarization Gabriel Shenouda1 Christophe Rodrigues1 Aurélien Bossard2 (1) Léonard De Vinci Pôle Universitaire, Research Center, 92 916 Paris La Défense, France (2) Laboratoire d’Informatique Avancée de Saint-Denis, Université Paris 8 (EA4383) 93200 Saint-Denis, France Abstract This paper introduces a new method, SummVD, for automatic unsupervised extractive summa- rization. This method is based on singular value decomposition, a linear method in the number of words, in order to reduce the dimensionality of word embeddings and propose a represen- tation of words on a small number of dimen- sions, each representing a hidden topic. It also uses word clustering to reduce the vocabulary size. This representation, specific to one doc- ument, reduces the noise brought by several dimensions of the embeddings that are useless in a restricted context. It is followed by a lin- ear sentence extraction heuristic. This makes SummVD an efficient method for text summa- rization. We evaluate SummVD using several corpora of different nature (news, scientific arti- cles, social network). Our method outperforms in effectiveness recent extractive approaches. Moreover, SummVD requires low resources, in terms of data and computing power. So it can be run on long single documents such as scien- tific papers as much as large multi-document corpora and is fast enough to be used in live summarization systems. 1 Introduction Research on automatic summarization has recently focused on supervised approaches. Since Pointer Generator by See et al. (2017), there has been con- siderable advances in the supervised generative summarization field (Zhang et al., 2020; Wu et al., 2021; Liu et al., 2021; Zhong et al., 2020). How- ever, these approaches need substantial learning corpora composed of a large amount of documents and summary pairs, and despite recent advances on fine-tuning and transfer learning, are limited to specific domains. Thus research on unsuper- vised summarization methods cannot be left out. In this paper, we tackle the problem of unsupervised extractive summarization, which aims to select sen- tences from one or multiple documents and put them together in order to build a summary. This extraction is often based on centrality and diversity notions : how much is a sentence central to the in- put text, and how many of the central information is present in the output summary. Inspired by the work of (Gong et al., 2018) on long texts similarity computation, we assume that hidden topics specific to a text can emerge from word embeddings computed from a general corpus. Each topic stands for a particular aspect of the text semantics. These hidden topics allow to remove un- necessary information from word representations and can be viewed as a new representation of the text. Words can be matched against a hidden topic, and this way, we can derive …

Content truncated to 500 words