Joint Inference for Mode Identification in Tutorial Dialogues

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2000–2011, Osaka, Japan, December 11-17 2016. Joint Inference for Mode Identification in Tutorial Dialogues Deepak Venugopal Department of Computer Science University of Memphis Memphis, TN 38152 dvngopal@memphis.edu Vasile Rus Department of Computer Science Institute for Intelligent Systems (IIS) University of Memphis Memphis, TN 38152 vrus@memphis.edu Abstract Identifying dialogue acts and dialogue modes during tutorial interactions is an extremely crucial sub-step in understanding patterns of effective tutor-tutee interactions. In this work, we develop a novel joint inference method that labels each utterance in a tutoring dialogue session with a dialogue act and a specific mode from a set of pre-defined dialogue acts and modes, respectively. Specifically, we develop our joint model using Markov Logic Networks (MLNs), a framework that combines first-order logic with probabilities, and is thus capable of representing complex, uncertain knowledge. We define first-order formulas in our MLN that encode the inter-dependencies between dialogue modes and more fine-grained dialogue actions. We then use a joint inference to jointly label the modes as well as the dialogue acts in an utterance. We compare our system against a pipeline system based on SVMs on a real-world dataset with tutoring sessions of over 500 students. Our results show that the joint inference system is far more effective than the pipeline system in mode detection, and improves over the performance of the pipeline system by about 6 points in F1 score. The joint inference system also performs much better than the pipeline system in the context of labeling modes that highlight important pedagogical steps in tutoring. 1 Introduction One-on-one instruction, i.e. tutoring, is one of the most effective forms of instruction. Intelligent Tutoring Systems (ITS) (Rus et al., 2013) have the potential to make effective and affordable “instruction-for-all” a reality since they do not suffer from traditional constraints such as lack of trained and expensive human tutors, physical teaching facilities, etc. However, in order to build effective automated tutoring systems, i.e. tutoring systems the induce student learning gains, we first need to understand what effective human tutors do. Specifically, we would like to identify specific pedagogical steps that promote effective tutoring. For instance, a good tutor may start by building a rapport with the students, followed by helping the student identify the domain of the problem, and so on. The sequence of steps taken by expert human tutors can in turn be used to improve the performance of ITS by re-enacting such effective tutorial strategies that are likely to promote better learning. Understanding what good tutors do to help students learn has been the subject of much theoretical and empirical research (Chi et al., 2001; Eugenio et al., 2006; Cade et al., 2008; Jeong et al., 2008; Boyer et al., 2010; Lehman et al., 2012). A standard approach to understanding effective tutoring is to characterize tutor-tutee interactions based on the actions tutors and tutees take and then identify patterns of such actions that are associated with effective tutoring. For …

Content truncated to 500 words