Automatic Generation and Classification of Minimal Meaningful Propositions in Educational Systems

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3226–3236, Osaka, Japan, December 11-17 2016. Automatic Generation and Classification of Minimal Meaningful Propositions in Educational Systems Andreea Godea, Florin Bulgarov and Rodney Nielsen Department of Computer Science and Engineering University of North Texas, TX, USA {AndreeaGodea, FlorinBulgarov}@my.unt.edu Rodney.Nielsen@colorado.edu Abstract Truly effective and practical educational systems will only be achievable when they have the ability to fully recognize deep relationships between a learner’s interpretation of a subject and the desired conceptual understanding. In this paper, we take important steps in this direction by introducing a new representation of sentences – Minimal Meaningful Propositions (MMPs), which will allow us to significantly improve the mapping between a learner’s answer and the ideal response. Using this technique, we make significant progress towards highly scalable and domain independent educational systems, that will be able to operate without human intervention. Even though this is a new task, we show very good results both for the extraction of MMPs and for classification with respect to their importance. 1 Introduction Over the last few decades, technology has provided us many powerful tools that have completely changed our daily routines. However, one crucial area where technology has yet to have the significant impact suggested by its true promise is in education. Most students around the world have been learning in the same manner for decades. Nevertheless, in the past few years technology has started to increase its role in the learning process and has begun improving the effectiveness of students and instructors. Several groups are developing tools or systems with the goal of improving the feedback provided to students and instructors, assessing students’ understanding of a concept, and facilitating their self-guided learning. Intelligent Tutoring Systems (ITSs) were created with the goal of improving learning through real- time and personalized feedback for students (Graesser et al., 2001; Ros´e et al., 2003; Makatchev et al., 2004; Pon-Barry et al., 2004). ITSs need to be able to interpret complex student responses and improve their feedback as they process more questions and responses, but essentially all existing systems require skilled developers to write new rules or train new classifiers for each additional question. Generally, an ITS only provides feedback to students, and when they do provide feedback to instructors, it is typically just high-level information regarding the correctness of the answer. Much of the prior work in this area originated in educational assessment systems (Mitchell et al., 2002; Sukkarieh et al., 2003; Nielsen et al., 2009). Most such systems investigate similarity or entailment relationships between a learner’s answer and the reference answer, and then communicate a score to the teacher. More recently, a new type of educational technology has emerged with the goal of increasing stu- dent engagement in classrooms (Paiva et al., 2014). Classroom Engagement Systems (CES) are meant to replace audience response (clicker) systems (Duncan, 2006; Fies and Marshall, 2006) by allowing students to construct answers to free-response questions. Unlike an ITS, the main goal of …

Content truncated to 500 words