ConvoSumm: Conversation Summarization Benchmark and Improved Abstractive Summarization with Argument Mining

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 6866–6880 August 1–6, 2021. ©2021 Association for Computational Linguistics 6866 ConvoSumm: Conversation Summarization Benchmark and Improved Abstractive Summarization with Argument Mining Alexander R. Fabbri[†] Faiaz Rahman[†] Imad Rizvi[†] Borui Wang[†] Haoran Li [‡] Yashar Mehdad[‡] Dragomir Radev[†] [†] Yale University [‡] Facebook AI {alexander.fabbri, faiaz.rahman, imad.rizvi, borui.wang, dragomir.radev}@yale.edu {aimeeli, mehdad}@fb.com Abstract While online conversations can cover a vast amount of information in many different for- mats, abstractive text summarization has pri- marily focused on modeling solely news ar- ticles. This research gap is due, in part, to the lack of standardized datasets for summa- rizing online discussions. To address this gap, we design annotation protocols motivated by an issues–viewpoints–assertions framework to crowdsource four new datasets on diverse on- line conversation forms of news comments, discussion forums, community question an- swering forums, and email threads. We bench- mark state-of-the-art models on our datasets and analyze characteristics associated with the data. To create a comprehensive benchmark, we also evaluate these models on widely-used conversation summarization datasets to estab- lish strong baselines in this domain. Fur- thermore, we incorporate argument mining through graph construction to directly model the issues, viewpoints, and assertions present in a conversation and filter noisy input, show- ing comparable or improved results according to automatic and human evaluations. 1 Introduction Automatic text summarization is the process of outputting the most salient parts of an input in a concise and readable form. Recent work in sum- marization has made significant progress due to introducing large-scale datasets such as the CNN- DailyMail dataset (Nallapati et al., 2016) and the New York Times dataset (Sandhaus, 2008). Further- more, the use of large self-supervised pretrained models such as BART (Lewis et al., 2020) and Pegasus (Zhang et al., 2019) has achieved state- of-the-art performance across summarization tasks and strong performance in zero and few-shot set- tings (Fabbri et al., 2020a). However, less work has focused on summarizing online conversations. Headline: SuperBowl Snippet: Whether you’re a football fan or not, what do you like about Super Bowl Sunday? Comment: ... In my opinion I think the Falcons will stomp the patriots. I think Tom Brady will choke the Super Bowl. ... Comment: I am big Arizona Cardinals fan so when they didn’t even make the playoffs i was upset. ... Comment: I’m not a very big football fan at all. So when it comes to Superbowl Sunday, I’m in it for the commercials and the half time show. ... Comment: I am not exactly a football fan, but I enjoy ...watching the Super Bowl.... Summary: Several commenters list their favorite things about the Super Bowl, including half-time shows, the funny com- mercials, the Puppy Bowl, eating food, and spending time with family. A couple of commenters admit to not being football fans but still enjoying the Super Bowl. Some com- menters discuss whether they thought the Falcons or the Patriots were …

Content truncated to 500 words