A Finite-State Turn-Taking Model for Spoken Dialog Systems

Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the ACL, pages 629–637, Boulder, Colorado, June 2009. c⃝2009 Association for Computational Linguistics A Finite-State Turn-Taking Model for Spoken Dialog Systems Antoine Raux∗ Honda Research Institute 800 California Street Mountain View, CA 94041, USA araux@hra.com Maxine Eskenazi Language Technologies Institute Carnegie Mellon University Pittsburgh, PA 15213, USA max@cs.cmu.edu Abstract This paper introduces the Finite-State Turn- Taking Machine (FSTTM), a new model to control the turn-taking behavior of conversa- tional agents. Based on a non-deterministic finite-state machine, the FSTTM uses a cost matrix and decision theoretic principles to se- lect a turn-taking action at any time. We show how the model can be applied to the problem of end-of-turn detection. Evaluation results on a deployed spoken dialog system show that the FSTTM provides significantly higher respon- siveness than previous approaches. 1 Introduction Turn-taking, the process by which participants in a conversation alternate speech and silence, is an es- sential component of spoken interaction. In order to lead productive conversations, people need not only know what to say but also when to say it. Decades of research on Conversation Analysis and psycholin- guistics (Duncan, 1972; Sacks et al., 1974; Ore- str¨om, 1983; Schegloff, 2000; Wesseling and van Son, 2005) have shown that human turn-taking be- havior relies on a wide range of rules and signals at many different levels of language, from prosody to syntax, semantics, and discourse structure. In contrast, turn-taking in spoken dialog systems is of- ten reduced to ad hoc rules only based on very low level features. This simplistic approach leads to in- efficient, unnatural, and possibly confusing behavior (Porzel and Baudis, 2004; Ward et al., 2005). ∗ This research was conducted when the first author was a student at the Language Technologies Institute. Recently, more complex models of turn-taking have been proposed (Cassell et al., 2001; Thorisson, 2002; Kronild, 2006). Yet, these models still rely extensively on hand-coded expert knowledge and do not lend themselves to data-driven optimization. Furthermore, to our knowledge, no such model has been deployed in a widely used system outside of the laboratory. In this paper, we propose a flexible, prac- tical model of turn-taking behavior that builds upon previous work on finite-state models of the conver- sational floor. Because of its simplicity and gener- ality, this model can be applied to many turn-taking phenomena. At the same time, being grounded in decision theory, it lends itself well to data-driven op- timization. We illustrate our approach by applying the model to a specific turn-taking task: end-of-turn detection. 2 Conversational Floor as a Finite-State Machine 2.1 6-state finite state models of turn-taking In the 1960’s and early 1970’s, several researchers proposed models to explain the rhythmic turn-taking patterns in human conversation. In particular, Jaffe and Feldstein (1970) studied the mean duration of pauses, switching pauses (when a different speaker takes the floor), simultaneous speech, and (single- speaker) vocalizations in recorded dyadic conversa- tions. Based on their observation that these dura- tions …

Content truncated to 500 words