Predicting Student Emotions in Computer-Human Tutoring Dialogues

Predicting Student Emotions in Computer-Human Tutoring Dialogues Diane J. Litman University of Pittsburgh Department of Computer Science Learning Research and Development Center Pittsburgh PA, 15260, USA litman@cs.pitt.edu Kate Forbes-Riley University of Pittsburgh Learning Research and Development Center Pittsburgh PA, 15260, USA forbesk@pitt.edu Abstract We examine the utility of speech and lexical fea- tures for predicting student emotions in computer- human spoken tutoring dialogues. We first anno- tate student turns for negative, neutral, positive and mixed emotions. We then extract acoustic-prosodic features from the speech signal, and lexical items from the transcribed or recognized speech. We com- pare the results of machine learning experiments us- ing these features alone or in combination to pre- dict various categorizations of the annotated student emotions. Our best results yield a 19-36% relative improvement in error reduction over a baseline. Fi- nally, we compare our results with emotion predic- tion in human-human tutoring dialogues. 1 Introduction This paper explores the feasibility of automatically predicting student emotional states in a corpus of computer-human spoken tutoring dialogues. Intel- ligent tutoring dialogue systems have become more prevalent in recent years (Aleven and Rose, 2003), as one method of improving the performance gap between computer and human tutors; recent exper- iments with such systems (e.g., (Graesser et al., 2002)) are starting to yield promising empirical results. Another method for closing this perfor- mance gap has been to incorporate affective reason- ing into computer tutoring systems, independently of whether or not the tutor is dialogue-based (Conati et al., 2003; Kort et al., 2001; Bhatt et al., 2004). For example, (Aist et al., 2002) have shown that adding human-provided emotional scaffolding to an auto- mated reading tutor increases student persistence. Our long-term goal is to merge these lines of dia- logue and affective tutoring research, by enhancing our intelligent tutoring spoken dialogue system to automatically predict and adapt to student emotions, and to investigate whether this improves learning and other measures of performance. Previous spoken dialogue research has shown that predictive models of emotion distinctions (e.g., emotional vs. non-emotional, negative vs. non- negative) can be developed using features typically available to a spoken dialogue system in real-time (e.g, acoustic-prosodic, lexical, dialogue, and/or contextual) (Batliner et al., 2000; Lee et al., 2001; Lee et al., 2002; Ang et al., 2002; Batliner et al., 2003; Shafran et al., 2003). In prior work we built on and generalized such research, by defin- ing a three-way distinction between negative, neu- tral, and positive student emotional states that could be reliably annotated and accurately predicted in human-human spoken tutoring dialogues (Forbes- Riley and Litman, 2004; Litman and Forbes-Riley, 2004). Like the non-tutoring studies, our results showed that combining feature types yielded the highest predictive accuracy. In this paper we investigate the application of our approach to a comparable corpus of computer- human tutoring dialogues, which displays many dif- ferent characteristics, such as shorter utterances, lit- tle student initiative, and non-overlapping speech. We investigate whether we can annotate and predict student emotions as accurately and whether the …

Content truncated to 500 words