Supporting Cognitive and Emotional Empathic Writing of Students

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 4063–4077 August 1–6, 2021. ©2021 Association for Computational Linguistics 4063 Supporting Cognitive and Emotional Empathic Writing of Students Thiemo Wambsganss1,2, Christina Niklaus1, 3, Matthias S¨ollner4, Siegfried Handschuh1, 3 and Jan Marco Leimeister1, 4 1 University of St.Gallen {thiemo.wambsganss, christina.niklaus, siegfried.handschuh, janmarco.leimeister}@unisg.ch 2 Carnegie Mellon University twambsga@andrew.cmu.edu 3 University of Passau {christina.niklaus, siegfried.handschuh}@uni-passau.de 4 University of Kassel {soellner, leimeister}@uni-kassel.de Abstract We present an annotation approach to cap- turing emotional and cognitive empathy in student-written peer reviews on business mod- els in German. We propose an annotation scheme that allows us to model emotional and cognitive empathy scores based on three types of review components. Also, we conducted an annotation study with three annotators based on 92 student essays to evaluate our annota- tion scheme. The obtained inter-rater agree- ment of α=0.79 for the components and the multi-π=0.41 for the empathy scores indicate that the proposed annotation scheme success- fully guides annotators to a substantial to mod- erate agreement. Moreover, we trained predic- tive models to detect the annotated empathy structures and embedded them in an adaptive writing support system for students to receive individual empathy feedback independent of an instructor, time, and location. We evalu- ated our tool in a peer learning exercise with 58 students and found promising results for perceived empathy skill learning, perceived feedback accuracy, and intention to use. Fi- nally, we present our freely available corpus of 500 empathy-annotated, student-written peer reviews on business models and our annotation guidelines to encourage future research on the design and development of empathy support systems. 1 Introduction Empathy is an elementary skill in society for daily interaction and professional communication and is therefore elementary for educational curricula (e.g., Learning Framework 2030 (OECD, 2018)). It is the “ability to simply understand the other person’s perspective [. . .] and to react to the ob- Figure 1: Empathy annotation scheme. First, a text paragraph is classified into a peer review component (strengths, weakness, improvement suggestions). Sec- ond, the same annotator is then scoring the cognitive and emotional empathy level of the components based on our annotation guideline on a 1-to-5 scale. served experiences of another,” (Davis, 1983, p.1)1. Empathy skills not only pave the foundation for successful interactions in digital companies, e.g., in agile work environments (Luca and Tarricone, 2001), but they are also one of the key abilities in the future that will distinguish the human work- force and artificial intelligence agents from one another (Poser and Bittner, 2020). However, be- sides the growing importance of empathy, research has shown that empathy skills of US college stu- dents decreased from 1979 to 2009 by more than thirty percent and even more rapidly between 2000 to 2009 (Konrath et al., 2011). On these grounds, the Organization for Economic Cooperation and Development (OECD) claims that the training for empathy skills should receive a more prominent role in today’s higher education …

Content truncated to 500 words