ExplainMeetSum: A Dataset for Explainable Meeting Summarization Aligned with Human Intent

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 13079–13098 July 9-14, 2023 ©2023 Association for Computational Linguistics ExplainMeetSum: A Dataset for Explainable Meeting Summarization Aligned with Human Intent Hyun Kim∗ and Minsoo Cho∗ Superintelligence Creative Research Lab., Electronics and Telecommunications Research Institute (ETRI), Republic of Korea {h.kim, mscho}@etri.re.kr Seung-Hoon Na† Computer Science and Engineering, Jeonbuk National University, Republic of Korea nash@jbnu.ac.kr Abstract To enhance the explainability of meeting sum- marization, we construct a new dataset called “ExplainMeetSum,” an augmented version of QMSum, by newly annotating evidence sen- tences that faithfully “explain” a summary. Us- ing ExplainMeetSum, we propose a novel mul- tiple extractor guided summarization, namely Multi-DYLE, which extensively generalizes DYLE to enable using a supervised extrac- tor based on human-aligned extractive ora- cles. We further present an explainability- aware task, named “Explainable Evidence Ex- traction” (E3), which aims to automatically de- tect all evidence sentences that support a given summary. Experimental results on the QMSum dataset show that the proposed Multi-DYLE outperforms DYLE with gains of up to 3.13 in the ROUGE-1 score. We further present the initial results on the E3 task, under the settings using separate and joint evaluation metrics.1 1 Introduction Meeting summarization typically is a form of long document summarization, because the input is usu- ally given as a long conversational sequence from multi-party dialogues. Among various approaches for long document summarization, the extract-then- generate method is one of the promising meth- ods; it first automatically selects “salient” contents which are relevant to a specific summarization and employs them to guide the generation of a sum- mary (Chen and Bansal, 2018; Zhang et al., 2019; Lebanoff et al., 2019; Xu and Durrett, 2019; Bajaj et al., 2021; Zhang et al., 2021; Mao et al., 2022), thereby inducing the manner of dealing with both efficiency (in processing a long input) and effec- tiveness (in locating accurately informative relevant contents). *These authors contributed equally to this work. †Corresponding author 1Our code and dataset are available at https://github. com/hkim-etri/ExplainMeetSum : Peripheral Evidence Sentence (PES) : Central Evidence Sentence (CES) (Sent1) The group discussed about … (Sent2) Speaker A stated … (Sent3) Speaker A also discussed … (Sent4) Speaker B presented … [Meeting Transcript] [Query] Summarize the discussion about … [Summary] • •• Speaker A : (Turn10) Speaker B : (Turn11) (10-0) (10-1) (10-2) Speaker C : (Turn13) (13-0) Speaker A : (Turn12) (12-0) (12-1) (12-2) (11-0) (11-2) (11-4) (11-1) (11-3) : Alignment between summary sentences and evidence sentences • • • Figure 1: An illustrated example of our annotation pro- cess in ExplainMeetSum where evidence sentences are manually aligned for every single summary sentence and categorized into two types – Central Evidence Sen- tence (CES) and Peripheral Evidence Sentence (PES), described in Section 3. However, the extract-then-generate method typ- ically selects salient content in a distantly super- vised or an end-to-end manner using only a final summary as a supervision signal, thereby likely being far from thoses in the …

Content truncated to 500 words