Hitting your MARQ: Multimodal ARgument Quality Assessment in Long Debate Video

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6387–6397 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 6387 Hitting your MARQ: Multimodal ARgument Quality Assessment in Long Debate Video Md Kamrul Hasan1, James Spann1, Masum Hasan1, Md Saiful Islam1, Kurtis Haut1, Rada Mihalcea 2, Ehsan Hoque1 1 - Department of Computer Science, University of Rochester, USA 2 - Computer Science & Engineering, University of Michigan, USA {mhasan8, jspann2}@cs.rochester.edu, {mhasan, mislam6, khaut}@ur.rochester.edu mihalcea@umich.edu, mehoque@cs.rochester.edu Abstract The combination of gestures, intonations, and textual content plays a key role in argument de- livery. However, the current literature mostly considers textual content while assessing the quality of an argument, and is limited to datasets containing short sequences (18-48 words). In this paper, we study argument qual- ity assessment in a multimodal context, and experiment on DBATES, a publicly available dataset of long debate videos. First, we pro- pose a set of interpretable debate-centric fea- tures such as clarity, content variation, body movement cues, and pauses, inspired by theo- ries of argumentation quality. Second, we de- sign the Multimodal ARgument Quality asses- sor (MARQ) – a hierarchical neural network model that summarizes the multimodal sig- nals on long sequences and enriches the multi- modal embedding with debate-centric features. Our proposed MARQ model achieves an accu- racy of 81.91% on the argument quality predic- tion task and outperforms established baseline models with an error rate reduction of 22.7%. Through ablation studies, we demonstrate the importance of multimodal cues in modeling ar- gument quality. 1 Introduction Structured debates and discussions are the basis for expressing opposing opinions, and are a tool for convincing others to share that opinion. Starting with a topic to argue, one can outline steps to reach a conclusion of why that topic is correct. This can take many forms in day-to-day life ranging from salesmen upselling a product or presidential de- bates, to people arguing whether to get vaccinated or to wear a mask. While the points of the argument may be valid, certain attributes such as clarity in the text, hand movements, and spoken style increase the effec- tiveness of the argument (Wachsmuth et al., 2017a; Braga and Marques, 2004; Straßmann et al., 2016). High Scoring Debater One Long Pause Low Scoring Debater Many Short Pauses Dynamic Body Movements Less Body Movements Figure 1: Delivering an argument involves multiple modalities. For example, a high scoring debater may use a certain prosodic style and dynamic hand gestures to present an argument more effectively, whereas a low scorer may hesitate (i.e., frequent short pauses) or show lack of confidence. Language only analysis fails to un- derstand these extra cues. These attributes increase the credibility of the speaker, and their ability to convince the listener (Figure 1). Measuring the quality of an argument given the language and other non-verbal features remains an elusive problem. Although argument quality assessment is an established research area in NLP, assessment in a multimodal context is un- derstudied. Most of the previous work focused on …

Content truncated to 500 words