A Bag of Tricks for Dialogue Summarization

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8014–8022 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 8014 A Bag of Tricks for Dialogue Summarization Muhammad Khalifa2∗, Miguel Ballesteros1, Kathleen McKeown1,3 1Amazon AI, USA 2Cairo University, Cairo, Egypt 3Department of Computer Science, Columbia University, NY, USA muhammad.e.khalifa@gmail.com {ballemig, mckeownk}@amazon.com Abstract Dialogue summarization comes with its own peculiar challenges as opposed to news or sci- entific articles summarization. In this work, we explore four different challenges of the task: handling and differentiating parts of the dialogue belonging to multiple speakers, nega- tion understanding, reasoning about the situ- ation, and informal language understanding. Using a pretrained sequence-to-sequence lan- guage model, we explore speaker name substi- tution, negation scope highlighting, multi-task learning with relevant tasks, and pretraining on in-domain data. Our experiments show that our proposed techniques indeed improve sum- marization performance, outperforming strong baselines. 1 Introduction The nature of dialogue poses additional challenges to summarizers beyond what is required when pro- cessing structured, single-speaker documents (Zhu and Penn, 2006). Given that dialogues typically represent an interaction between many speakers, a summarizer model must keep track of the differ- ent lines of thoughts of individual speakers, dis- tinguish salient from non-salient utterances, and finally produce a coherent, monologue summary of the dialogue. Dialogues usually include unfinished sentences where speakers were interrupted or repetitions, where a speaker expresses their thoughts more than once and possibly in different styles. Moreover, a single dialogue could touch on many topics without a clear boundary between the different topics. All the aforementioned phenomena certainly add to the difficulty of the task (Zechner and Waibel, 2000; Zechner, 2002; Chen and Yang, 2020). Our work focuses on SAMSum (Gliwa et al., 2019), which is a dialogue summarization dataset comprised of ~16K everyday dialogues with their ∗Work done during an internship at Amazon. human-written summaries. As our backbone model, we use BART (Lewis et al., 2020), a state- of-the-art pretrained encoder-decoder language model that is suitable for sequence-to-sequence tasks. Table 1 shows an example of a summary generated using BART (Lewis et al., 2020), fine- tuned on SAMSum. Clearly, a level of reasoning is required to make sense of the conversation, which BART fails to do and therefore produces an incor- rect summary. We propose a combination of techniques to tackle a set of dialogue summarization challenges. The first challenge is having multiple speakers (generally, more than 2), where it becomes harder for the model to keep track of different utterances and determine their saliency. The second challenge is multiple negations, which is thought by Chen and Yang (2020) to pose some difficulty to dia- logue understanding. The third of these challenges is reasoning, where the model is required to rea- son about the dialogue context, and infer informa- tion that is not explicitly expressed. The last chal- lenge is informal language. Since we focus on random, everyday conversations, these are usually filled with non-standard language (abbreviations, social media terms, etc.). The contributions in this work …

Content truncated to 500 words