MeetingBank: A Benchmark Dataset for Meeting Summarization

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 16409–16423 July 9-14, 2023 ©2023 Association for Computational Linguistics MeetingBank: A Benchmark Dataset for Meeting Summarization Yebowen Hu,† Tim Ganter,‡ Hanieh Deilamsalehy,‡ Franck Dernoncourt,‡ Hassan Foroosh,† Fei Liu§ †University of Central Florida ‡Adobe Research §Emory University huye@knights.ucf.edu hassan.foroosh@ucf.edu {ganter,deilamsa,franck.dernoncourt}@adobe.com fei.liu@emory.edu Abstract As the number of recorded meetings increases, it becomes increasingly important to utilize summarization technology to create useful sum- maries of these recordings. However, there is a crucial lack of annotated meeting corpora for developing this technology, as it can be hard to collect meetings, especially when the topics dis- cussed are confidential. Furthermore, meeting summaries written by experienced writers are scarce, making it hard for abstractive summariz- ers to produce sensible output without a reliable reference. This lack of annotated corpora has hindered the development of meeting summa- rization technology. In this paper, we present MeetingBank, a new benchmark dataset of city council meetings over the past decade. Meeting- Bank is unique among other meeting corpora due to its divide-and-conquer approach, which involves dividing professionally written meet- ing minutes into shorter passages and aligning them with specific segments of the meeting. This breaks down the process of summarizing a lengthy meeting into smaller, more manageable tasks. The dataset provides a new testbed of var- ious meeting summarization systems and also allows the public to gain insight into how coun- cil decisions are made. We make the collection, including meeting video links, transcripts, ref- erence summaries, agenda, and other metadata, publicly available to facilitate the development of better meeting summarization techniques.1 1 Introduction An astonishing 55 million meetings happen in the U.S. each week (Flynn, 2022). With the extensive use of video conferencing software, e.g., Microsoft Teams, Google Meet and Zoom, it has become eas- ier than ever before to record meetings. While these recordings provide a wealth of human intelligence and actionable knowledge, the temporal nature of sound makes it difficult for users to navigate and 1Our dataset can be accessed at: meetingbank.github.io search for specific content (Bengio and Bourlard, 2004). A summarization system that produces text summaries from transcripts can help, by providing users with great flexibility in navigating recordings, including but not limited to: meetings, interviews, podcasts, lectures, movies and TV series (Papalam- pidi et al., 2020; Zhu et al., 2021; Song et al., 2022; Chen et al., 2022; Cho et al., 2022). Effective meeting summarization requires anno- tated datasets. Most summarizers, including few- shot and prompt-based (Goyal et al., 2022), will benefit directly from benchmark datasets contain- ing hundreds of thousands of document-summary pairs such as XSum (Narayan et al., 2018), Multi- News (Fabbri et al., 2019), GovReport (Huang et al., 2021), PLoS (Goldsack et al., 2022). How- ever, datasets for meeting summarization are rela- tively scarce, small, or unrepresentative. ICSI and AMI are two benchmark datasets (Janin et al., 2003; Carletta et al., 2006) that consist of only 75 and 140 meetings, respectively. Other existing datasets for …

Content truncated to 500 words