Learning to Generate Overlap Summaries through Noisy Synthetic Data

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11765 - 11777 December 7-11, 2022 ©2022 Association for Computational Linguistics Learning to Generate Overlap Summaries through Noisy Synthetic Data Naman Bansal, Mousumi Akter and Shubhra Kanti Karmaker (“Santu”) Big Data Intelligence (BDI) Lab Department of Computer Science and Software Engineering College of Engineering, Auburn University {nbansal, mza0170, sks0086}@auburn.edu Abstract Semantic Overlap Summarization (SOS) is a novel and relatively under-explored seq-to-seq task which entails summarizing common in- formation from multiple alternate narratives. One of the major challenges for solving this task is the lack of existing datasets for super- vised training. To address this challenge, we propose a novel data augmentation technique, which allows us to create large amount of synthetic data for training a seq-to-seq model that can perform the SOS task. Through ex- tensive experiments using narratives from the news domain, we show that the models fine- tuned using the synthetic dataset provide sig- nificant performance improvements over the pre-trained vanilla summarization techniques and are close to the models fine-tuned on the golden training data; which essentially demon- strates the effectiveness of out proposed data augmentation technique for training seq-to-seq models on the SOS task. 1 Introduction Semantic Overlap Summarization (SOS) is a novel and relatively under-explored seq-to-seq task. It refers to the process of generating a summary of common information from multiple alternative nar- ratives originating from various sources. Figure 1 depicts a toy use-case of SOS task on two alterna- tive narratives (Bansal et al., 2022). Here, both arti- cles cover the same event related to a supreme court ruling on abortion in Kentucky, but from two oppo- site political viewpoints; one from the left wing and the other from the right wing (color coded). The goal of the SOS task is to generate the overlapping information (in green text). In this sense, the SOS task can be defined as a multi-seq-to-seq task with an additional overlap constraint. Multiple alternative narratives are frequent in a variety of domains, including education, health, and privacy, and summarizing the common infor- mation from these alternative narratives can be highly useful for digesting those multi-narratives at scale and speed (Karmaker Santu et al., 2018). However, one of the major challenges associated with implementing a seq-to-seq model which can perform the SOS task is the lack of readily avail- able training data for supervised learning. One may manually create a training corpus for a partic- ular domain (e.g., news, health etc.) by spending a significant amount of time and money, yet it is un- clear how much it will generalize for other domains. Therefore, an unsupervised approach is desired to address this problem. In this paper, we propose a new unsupervised data generation technique which can generate an arbitrarily large number of synthetic training ex- amples for the SOS task. More specifically, given an arbitrary text corpus from a particular domain, our data generation algorithm can produce an infinite number of SOS examples of the form {{DA, DB}, …

Content truncated to 500 words