QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5905–5921 June 6–11, 2021. ©2021 Association for Computational Linguistics 5905 QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization Ming Zhong∗† Da Yin∗♣ Tao Yu‡ Ahmad Zaidi‡ Mutethia Mutuma‡ Rahul Jha¶ Ahmed Hassan Awadallah¶ Asli Celikyilmaz¶ Yang Liu§ Xipeng Qiu† Dragomir Radev‡ †Fudan University ♣University of California, Los Angeles ‡Yale University ¶Microsoft Research §Microsoft Cognitive Services Research mzhong18@fudan.edu.cn da.yin@cs.ucla.edu {tao.yu, dragomir.radev}@yale.edu Abstract Meetings are a key component of human col- laboration. As increasing numbers of meetings are recorded and transcribed, meeting sum- maries have become essential to remind those who may or may not have attended the meet- ings about the key decisions made and the tasks to be completed. However, it is hard to create a single short summary that covers all the content of a long meeting involving multi- ple people and topics. In order to satisfy the needs of different types of users, we define a new query-based multi-domain meeting sum- marization task, where models have to select and summarize relevant spans of meetings in response to a query, and we introduce QMSum, a new benchmark for this task. QMSum con- sists of 1,808 query-summary pairs over 232 meetings in multiple domains. Besides, we in- vestigate a locate-then-summarize method and evaluate a set of strong summarization base- lines on the task. Experimental results and manual analysis reveal that QMSum presents significant challenges in long meeting summa- rization for future research. Dataset is avail- able at https://github.com/Yale-LILY/ QMSum. 1 Introduction Meetings remain the go-to tool for collaboration, with 11 million meetings taking place each day in the USA and employees spending six hours a week, on average, in meetings (Mroz et al., 2018). The emerging landscape of remote work is mak- ing meetings even more important and simultane- ously taking a toll on our productivity and well- being (Spataro, 2020). The proliferation of meet- ings makes it hard to stay on top of this sheer volume of information and increases the need for automated methods for accessing key informa- tion exchanged during them. Meeting summariza- tion (Wang and Cardie, 2013; Shang et al., 2018; ∗ These two authors contributed equally. The order of authorship decided by the flip of a coin. Meeting�Transcript Summarize�the�whole�meeting. The�meeting�was�mainly�related�to�...... Turn�0: Project�Manager:�We�have�been�provided�with�some technical� tools�to�communicate. ...... ...... ...... ...... Turn�316:�Project�Manager:�Thanks.�Have�a�nice�day! Summarize�the�discussion about�the�trends�of�current remote�controls. The�group�discussed�different trends�based�on�different�ages�of people.�......�Finally�they�decided�to add�LCD�screen.� What�did�User�Interface�Designer think�of�surface�design�when discussing�user�interface?� User�Interface�Designer�said�the remote�should�perform�standard features�right�out-of-the-box�...... Turn�16:�Marketing:�This�is�just�a�presentation�on the�trends�that�we're�gonna�use�to�make�the�product�stand� out�from�...... ...... Turn�78:�Marketing:�Young�people�like�that�things�with� cool�appearance. Turn�85:�Marketing:�What�do�you�think�of�adding�an�LCD? ...... Turn�89:�Project�Manager:�Okay,�we'll�include�it�to�make� the�appearance�attractive�to�young�people. Turn�121:�User�Interface�Designer:�The�idea�of�having�a� remote�is�you�have�different�keys�and�different�structures. ...... Turn�162:�Project�Manager:�Sure.�Let's�push�forward�the� interface�design. Figure 1: Examples of query-based meeting summa- rization task. Users are interested in different facets of the meeting. In this task, a model is required to summa- rize the contents that users are interested in and query. Li et al., 2019; Zhu et al., 2020) is a task where summarization models are leveraged to generate summaries of entire meetings based on meeting transcripts. The resulting summaries distill the core contents of a meeting that helps people efficiently …

Content truncated to 500 words