Modeling Persuasive Discourse to Adaptively Support Students’ Argumentative Writing

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 8748 - 8760 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Modeling Persuasive Discourse to Adaptively Support Students’ Argumentative Writing Thiemo Wambsganss University of St.Gallen/ CH Carnegie Mellon University/ US thiemo.wambsganss@unisg.ch Christina Niklaus University of St.Gallen/ CH christina.niklaus@unisg.ch Abstract We introduce an argumentation annotation ap- proach to model the structure of argumentative discourse in student-written business model pitches. Additionally, the annotation scheme captures a series of persuasiveness scores such as the specificity, strength, evidence, and rel- evance of the pitch and the individual compo- nents. Based on this scheme, we annotated a corpus of 200 business model pitches in Ger- man. Moreover, we trained predictive models to detect argumentative discourse structures and embedded them in an adaptive writing sup- port system for students that provides them with individual argumentation feedback inde- pendent of an instructor, time, and location. We evaluated our tool in a real-world writing exercise and found promising results for the measured self-efficacy and perceived ease-of- use. Finally, we present our freely available corpus of persuasive business model pitches with 3,207 annotated sentences in German lan- guage and our annotation guidelines. 1 Introduction Argumentation is an omnipresent rudiment of daily communication and thinking (Kuhn, 1992; Toul- min, 1984). The ability to form convincing ar- guments is not only fundamental to persuading an audience of novel ideas but also plays a major role in strategic decision-making, negotiation, and constructive civil discourse (Walton et al., 2008; Scheuer et al., 2010). However, humans often struggle to develop argumentation skills owing to a lack of individual and instant feedback in their learning process (Dillenbourg et al., 2009; Hattie and Timperley, 2007), since providing feedback on the individual argumentation skills of learners is time-consuming and not scalable if conducted manually by educators (OECD, 2018; Wambsganss et al., 2020b). Furthermore, novel distance learn- ing scenarios such as massive open online courses (MOOCs) (Seaman et al., 2018) come with addi- Figure 1: Argumentation annotation scheme. First, a text sentence is classified into an argumenta- tive component (claim, premise, major claim, none- argumentative). Second, the same annotator captures the basic discourse structure between the components. Third, the components and the pitch are scored for the persuasiveness scores (specificity, evidence, strength, relevance) based on our annotation guideline on a 1- to-5 scale. tional barriers related to individual feedback on a learner’s argumentation. One possible solution to this dilemma are adap- tive argumentation support systems that enable in- dividuals to train their argumentation skills, e.g., in collaborative learning settings (Dillenbourg et al., 2009) or by providing tailored argumentation feed- back independent of an instructor, time and place (Wambsganss et al., 2020b, 2021). Such tools are increasingly utilizing recent developments in computational linguistics in the form of computer- assisted writing (Rosé et al., 2008) to provide tailored feedback about textual documents (Song et al., 2014; Stab and Gurevych, 2014a). In this context, Argumentation Mining (AM) research is a crucial field for the …

Content truncated to 500 words