Discovering Latent Structure in Task-Oriented Dialogues

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 36–46, Baltimore, Maryland, USA, June 23-25 2014. c⃝2014 Association for Computational Linguistics Discovering Latent Structure in Task-Oriented Dialogues Ke Zhai∗ Computer Science, University of Maryland College Park, MD 20740 zhaike@cs.umd.edu Jason D. Williams Microsoft Research Redmond, WA 98052 jason.williams@microsoft.com Abstract A key challenge for computational conver- sation models is to discover latent struc- ture in task-oriented dialogue, since it pro- vides a basis for analysing, evaluating, and building conversational systems. We pro- pose three new unsupervised models to discover latent structures in task-oriented dialogues. Our methods synthesize hidden Markov models (for underlying state) and topic models (to connect words to states). We apply them to two real, non-trivial datasets: human-computer spoken dia- logues in bus query service, and human- human text-based chats from a live tech- nical support service. We show that our models extract meaningful state represen- tations and dialogue structures consistent with human annotations. Quantitatively, we show our models achieve superior per- formance on held-out log likelihood eval- uation and an ordering task. 1 Introduction Modeling human conversation is a fundamental scientific pursuit. In addition to yielding ba- sic insights into human communication, compu- tational models of conversation underpin a host of real-world applications, including interactive dialogue systems (Young, 2006), dialogue sum- marization (Murray et al., 2005; Daum´e III and Marcu, 2006; Liu et al., 2010), and even medi- cal applications such as diagnosis of psychological conditions (DeVault et al., 2013). Computational models of conversation can be broadly divided into two genres: modeling and control. Control is concerned with choosing ac- tions in interactive settings—for example to maxi- mize task completion—using reinforcement learn- ∗Work done at Microsoft Research. ing (Levin et al., 2000), supervised learning (Hur- tado et al., 2010), hand-crafted rules (Larsson and Traum, 2000), or mixtures of these (Henderson and Lemon, 2008). By contrast, modeling—the genre of this paper—is concerned with inferring a phenomena in an existing corpus, such as di- alogue acts in two-party conversations (Stolcke et al., 2000) or topic shifts in multi-party dia- logues (Galley et al., 2003; Purver et al., 2006; Hsueh et al., 2006; Banerjee and Rudnicky, 2006). Many past works rely on supervised learning or human annotations, which usually requires man- ual labels and annotation guidelines (Jurafsky et al., 1997). It constrains scaling the size of training examples, and application domains. By contrast, unsupervised methods operate only on the observ- able signal (e.g. words) and are estimated with- out labels or their attendant limitations (Crook et al., 2009). They are particularly relevant because conversation is a temporal process where models are trained to infer a latent state which evolves as the dialogue progresses (Bangalore et al., 2006; Traum and Larsson, 2003). Our basic approach is to assume that each ut- terance in the conversation is in a latent state, which has a causal effect on the words the conver- sants produce. Inferring this model yields basic insights into the structure of conversation and also has broad practical …

Content truncated to 500 words