MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarization

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5927–5934 June 6–11, 2021. ©2021 Association for Computational Linguistics 5927 MEDIASUM: A Large-scale Media Interview Dataset for Dialogue Summarization Chenguang Zhu∗, Yang Liu∗, Jie Mei, Michael Zeng Microsoft Cognitive Services Research Group {chezhu,yaliu10,jimei,nzeng}@microsoft.com Abstract This paper introduces MEDIASUM1, a large- scale media interview dataset consisting of 463.6K transcripts with abstractive summaries. To create this dataset, we collect interview transcripts from NPR and CNN and employ the overview and topic descriptions as sum- maries. Compared with existing public cor- pora for dialogue summarization, our dataset is an order of magnitude larger and contains complex multi-party conversations from multi- ple domains. We conduct statistical analysis to demonstrate the unique positional bias exhib- ited in the transcripts of televised and radioed interviews. We also show that MEDIASUM can be used in transfer learning to improve a model’s performance on other dialogue sum- marization tasks. 1 Introduction Dialogue summarization can provide a suc- cinct synopsis for conversations between two or more participants, based on human-transcribed or machine-generated transcripts. Dialogue sum- maries are useful for participants to recap salient information in the talk and for absentees to grasp the key points. As a result, several models have been recently proposed to summarize daily conver- sations (Gliwa et al., 2019; Chen and Yang, 2020), meeting transcripts (Zhu et al., 2020) and customer support conversations (Liu et al., 2019). However, compared with the abundance of text summarization datasets, there are very few public datasets for dialogue summarization. And exist- ing datasets are limited to their small sizes. For example, the benchmark datasets for meeting sum- marization, AMI (McCowan et al., 2005) and ICSI (Janin et al., 2003), only contain transcripts and abstractive summaries for 137 and 59 business ∗Equal contribution 1https://github.com/zcgzcgzcg1/ MediaSum/ meetings, respectively. While recently some larger dialogue summarization datasets have been pro- posed, they are either built from a narrow domain, e.g. the CRD3 dataset (Rameshkumar and Bailey) which is built from conversations in a live-streamed show for the Dungeons and Dragons game, or not publicized due to privacy reasons, e.g. the Didi dataset (Liu et al., 2019) from customer service conversations. This lack of large-scale dialogue summarization datasets is due to a higher labeling cost compared with news articles and privacy is- sues with many real daily dialogues and business meetings. On the other hand, media interview transcripts and the associated summaries/topics can be a valu- able source for dialogue summarization. In a broad- cast interview, the host discusses various topics with one or more guests. As many interviews pro- ceed with pre-defined topics, the accompanying summaries are of a relatively high quality. Also, the wide variety of topics, different backgrounds of speakers, and the colloquial form of chat make these interviews very close to daily conversations and business meetings. Therefore, we collect public interview tran- scripts and the associated summaries/topics from NPR and CNN to build a large-scale dialogue sum- marization dataset, MEDIASUM. …

Content truncated to 500 words