Abstractive Document Summarization without Parallel Data

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 6638–6644 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 6638 Abstractive Document Summarization without Parallel Data Nikola I. Nikolov and Richard H.R. Hahnloser Institute of Neuroinformatics, University of Z¨urich and ETH Z¨urich, Switzerland {niniko, rich}@ini.ethz.ch Abstract Abstractive summarization typically relies on large collections of paired articles and summaries. However, in many cases, parallel data is scarce and costly to obtain. We develop an abstractive summarization system that relies only on large collections of example summaries and non-matching articles. Our approach consists of an unsupervised sentence extractor that selects salient sentences to include in the final summary, as well as a sentence abstractor that is trained on pseudo-parallel and synthetic data, that paraphrases each of the extracted sentences. We perform an extensive evaluation of our method: on the CNN/DailyMail benchmark, on which we compare our approach to fully supervised baselines, as well as on the novel task of automatically generating a press release from a scientific journal article, which is well suited for our system. We show promising performance on both tasks, without relying on any article-summary pairs. Keywords: automatic summarization, abstractive summarization, text rewriting, low-resource settings 1. Introduction Text summarization aims to produce a shorter, informa- tive version of an input text. While extractive summariza- tion only selects important sentences from the input, ab- stractive summarization generates content without explic- itly re-using whole sentences (Nenkova et al., 2011) re- sulting summaries that are more fluent. In recent years, a number of successful approaches have been proposed for both extractive (Nallapati et al., 2017; Narayan et al., 2018) and abstractive (See et al., 2017; Chen and Bansal, 2018) summarization paradigms. State-of-the-art abstractive ap- proaches are supervised, relying on large collections of paired articles and summaries. However, competitive per- formance of abstractive systems remains a challenge when the availability of parallel data is limited, such as in low- resource domains or for languages other than English. Even when parallel data is severely limited, we may have access to example summaries and large collections of arti- cles on similar topics. Examples are blog posts or scientific press releases, for which the original articles may be un- available or behind a paywall. In this paper, we develop a system1 for abstractive docu- ment summarization (Figure 1) that only relies on example summaries and non-matching articles, bypassing the need for large-scale parallel corpora. Our system consists of two components: First, an unsupervised sentence extractor se- lects salient sentences. Second, each extracted sentence is paraphrased using a sentence abstractor. The abstractor is trained on pseudo-parallel data extracted from raw corpora, as well as on additional synthetic data generated through backtranslation. We evaluate our approach on two summarization tasks. First, on the CNN/DailyMail news article summarization dataset, we compare the performance of our method based on non-parallel data to fully supervised models based on parallel data (Section 5.1.). Second, we test our system on the novel task of automatically …

Content truncated to 500 words