Predicting Success in Dialogue

Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 808–815, Prague, Czech Republic, June 2007. c⃝2007 Association for Computational Linguistics Predicting Success in Dialogue David Reitter and Johanna D. Moore dreitter | jmoore @ inf.ed.ac.uk School of Informatics University of Edinburgh United Kingdom Abstract Task-solving in dialogue depends on the lin- guistic alignment of the interlocutors, which Pickering & Garrod (2004) have suggested to be based on mechanistic repetition ef- fects. In this paper, we seek confirmation of this hypothesis by looking at repetition in corpora, and whether repetition is cor- related with task success. We show that the relevant repetition tendency is based on slow adaptation rather than short-term prim- ing and demonstrate that lexical and syntac- tic repetition is a reliable predictor of task success given the first five minutes of a task- oriented dialogue. 1 Introduction While humans are remarkably efficient, flexible and reliable communicators, we are far from perfect. Our dialogues differ in how successfully informa- tion is conveyed. In task-oriented dialogue, where the interlocutors are communicating to solve a prob- lem, task success is a crucial indicator of the success of the communication. An automatic measure of task success would be useful for evaluating conversations among humans, e.g., for evaluating agents in a call center. In human- computer dialogues, predicting the task success after just a first few turns of the conversation could avoid disappointment: if the conversation isn’t going well, a caller may be passed on to a human operator, or the system may switch dialogue strategies. As a first step, we focus on human-human dialogue, since cur- rent spoken dialogue systems do not yet yield long, syntactically complex conversations. In this paper, we use syntactic and lexical features to predict task success in an environment where we assume no speaker model, no semantic information and no information typical for a human-computer dialogue system, e.g., ASR confidence. The fea- tures we use are based on a psychological theory, linking alignment between dialogue participants to low-level syntactic priming. An examination of this priming reveals differences between short-term and long-term effects. 1.1 Repetition supports dialogue In their Interactive Alignment Model (IAM), Pick- ering and Garrod (2004) suggest that dialogue be- tween humans is greatly aided by aligning repre- sentations on several linguistic and conceptual lev- els. This effect is assumed to be driven by a cas- cade of linguistic priming effects, where interlocu- tors tend to re-use lexical, syntactic and other lin- guistic structures after their introduction. Such re- use leads speakers to agree on a common situa- tion model. Several studies have shown that speak- ers copy their interlocutor’s syntax (Branigan et al., 1999). This effect is usually referred to as structural (or: syntactic) priming. These persistence effects are inter-related, as lexical repetition implies pref- erences for syntactic choices, and syntactic choices lead to preferred semantic interpretations. Without demanding additional cognitive resources, the ef- fects form a causal chain that will benefit the inter- locutor’s purposes. Or, at the very least, …

Content truncated to 500 words