An Exploration of Off Topic Conversation

Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL, pages 669–672, Los Angeles, California, June 2010. c⃝2010 Association for Computational Linguistics An Exploration of Off Topic Conversation Whitney L. Cade University of Memphis 365 Innovation Drive Memphis, TN 38152-3115 wlcade@memphis.edu Blair A. Lehman University of Memphis 365 Innovation Drive Memphis, TN 38152-3115 baleh- man@memphis.edu Andrew Olney University of Memphis 365 Innovation Drive Memphis, TN 38152-3115 aolney@memphis.edu Abstract In a corpus of expert tutoring dialogue, con- versation that is considered to be “off topic” (non-pedagogical) according to a previous coding scheme is explored for its value in tu- toring dynamics. Using the Linguistic Inquiry and Word Count (LIWC) tool, phases of tutor- ing categorized as “off topic” were compared with interactive problem solving phases to ex- plore how the two differ on the emotional, psychological, and topical dimensions ana- lyzed by LIWC. The results suggest that con- versation classified as “off topic” serves as motivation and broad pedagogy in tutoring. These findings can be used to orient future re- search on “off topic” conversation, and help to make sense of both previous coding schemes and noisy data sets. 1 Introduction Methods of investigating a large and noisy data set are of paramount importance in computational lin- guistics. Quite often, qualitative coding schemes are used to capture snapshots of the data set, but these may gloss over finer details or miss the larger picture. Add to that the messy and unpredictable nature of naturalistic data, and analysis becomes even more complicated. Therefore, a multi-method approach to understanding pre-existing coding schemes and orienting future in-depth analyses of those schemes proves to be a useful means of ex- ploring one’s data. Dialogue, particularly tutorial dialogue, is one area where large, noisy data sets are common. Computer and human tutoring data have been parsed, coded, and tested by a number of research- ers, and much effort has been put into making sense of the variability in the task-oriented dialo- gue (e.g. Chi, Roy, and Hausmann, 2008; Graesser, Person, and Magliano, 1995; Person, Lehman, and Ozbun, 2007). This work has all been in pursuit of a deep understanding of the complex interaction between the human tutor and student, which, if understood, could be used to boost the efficacy of artificially intelligent computer tutors. Expert hu- man tutoring has been found to increase learning gains by as much as 2 sigmas (Bloom, 1984), which makes understanding their methods and mo- tives the goal of any tutor research. The corpus under examination here was col- lected with the express purpose of understanding how truly expert tutors manage a tutoring session, with an emphasis on creating a corpus of naturalis- tic dialogue data. The corpus has been investigated at two different grain sizes, a dialogue move level and a sustained phases level. Our study investi- gates in detail an “other” category that these cod- ing schemes, which emphasize the pedagogy of the tutors and the students reactions, classify as “off topic” conversation. Off …

Content truncated to 500 words