Simple Conversational Data Augmentation for Semi-supervised Abstractive Dialogue Summarization

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6605–6616 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 6605 Simple Conversational Data Augmentation for Semi-supervised Abstractive Conversation Summarization Jiaao Chen School of Interactive Computing Georgia Institute of Technology jiaaochen@gatech.edu Diyi Yang School of Interactive Computing Georgia Institute of Technology dyang888@gatech.edu Abstract Abstractive conversation summarization has received growing attention while most cur- rent state-of-the-art summarization models heavily rely on human-annotated summaries. To reduce the dependence on labeled sum- maries, in this work, we present a sim- ple yet effective set of Conversational Data Augmentation (CODA) methods for semi- supervised abstractive conversation summa- rization, such as random swapping/deletion to perturb the discourse relations inside con- versations, dialogue-acts-guided insertion to interrupt the development of conversations, and conditional-generation-based substitution to substitute utterances with their paraphrases generated based on the conversation context. To further utilize unlabeled conversations, we combine CODA with two-stage noisy self- training where we first pre-train the sum- marization model on unlabeled conversations with pseudo summaries and then fine-tune it on labeled conversations. Experiments con- ducted on the recent conversation summariza- tion datasets demonstrate the effectiveness of our methods over several state-of-the-art data augmentation baselines. We have publicly re- leased our code at https://github.com/ GT-SALT/CODA. 1 Introduction Abstractive conversation summarization, which tar- gets at processing, organizing and distilling human interaction activities into short, concise and nat- ural text (Murray et al., 2006; Wang and Cardie, 2013), is one of the most challenging and inter- esting problems in text summarization. Recently, neural abstractive conversation summarization has received growing attention and achieved remark- able performances by adapting document summa- rization pre-trained models and (Gliwa et al., 2019; Yu et al., 2021) and incorporating structural infor- mation (Chen and Yang, 2020; Feng et al., 2020c; Zhu et al., 2020a; Chen and Yang, 2021; Liu et al., 2019b). However, most of these models usually require abundant human-annotated summaries to yield the state-of-the-art performances (Gliwa et al., 2019), making them hard to be applied into real- world applications (e.g. summarizing counseling sessions) that lack labeled summaries. Data augmentation, which perturbs input data to create additional augmented data, has been uti- lized to alleviate the need of labeled data in various NLP tasks, and can be categorized into three ma- jor classes: (1) manipulating words and phrases at the token-level like designed word replacement (Kobayashi, 2018; Niu and Bansal, 2018), word deletion/swapping/insertion (Wei and Zou, 2019; Feng et al., 2020a), token/span cutoff (Shen et al., 2020b); (2) paraphrasing the entire input text at the sentence-level through round-trip translation (Sennrich et al., 2015; Xie et al., 2019; Chen et al., 2020b) or syntactic manipulation (Iyyer et al., 2018; Chen et al., 2020c); and (3) adding adversar- ial perturbations to the original data which dramat- ically influences the model’s predictions (Jia and Liang, 2017; Niu and Bansal, 2019; Zhang et al., 2019). Despite the huge success, the former two mainly perturbs sentences locally while ignoring the diverse structures and context information in dialogues to create high-quality …

Content truncated to 500 words