Statistical Dialogue Management using Intention Dependency Graph

International Joint Conference on Natural Language Processing, pages 962–966, Nagoya, Japan, 14-18 October 2013. Statistical Dialogue Management using Intention Dependency Graph Koichiro Yoshino1,2, Shinji Watanabe1, Jonathan Le Roux1, John R. Hershey1 1Mitsubishi Electric Research Laboratories, 201 Broadway, Cambridge, MA, 02139, USA {watanabe,leroux,hershey}@merl.com 2School of Informatics, Kyoto University, Sakyo, Kyoto, 606-8501, Japan yoshino@ar.media.kyoto-u.ac.jp Abstract We present a method of statistical dia- logue management using a directed inten- tion dependency graph (IDG) in a par- tially observable Markov decision pro- cess (POMDP) framework. The transition probabilities in this model involve infor- mation derived from a hierarchical graph of intentions. In this way, we combine the deterministic graph structure of a con- ventional rule-based system with a statis- tical dialogue framework. The IDG also provides a reasonable constraint on a user simulation model, which is used when learning a policy function in POMDP and dialogue evaluation. Thus, this method converts a conventional dialogue manager to a statistical dialogue manager that uti- lizes task domain knowledge without an- notated dialogue data. 1 Introduction Statistical approaches based on reinforcement learning, such as the Markov decision process (MDP) and partially observable Markov decision process (POMDP), have been successfully ap- plied to dialogue management (Levin et al., 2000; Williams and Young, 2007; Li, 2012). These ap- proaches allow us to consider all possible future actions of a dialogue system, and thus to obtain a new optimal dialogue strategy which could not be anticipated in conventional hand-crafted dialogue systems. Moreover, the statistical dialogue frame- work can be combined with conventional rule- based dialogue management in hybrid systems, (Williams, 2008; Lee et al., 2010), which com- bine the optimal dialogue strategy in the statistical approach with the lower cost of data and mainte- nance of the rule-based approach. Our research focuses on a practical application of a hybrid statistical dialogue management based on POMDP to conventional rule-based dialogue management via the use of an intention depen- dency graph (IDG). The IDG derives from the con- ventional rule-based dialogue system (Dahl et al., 1994; Bohus and Rudnicky, 2003), and it con- strains the transition matrix and provides a user simulation as a substitute for dialogue data. The object of POMDP optimization is to pro- duce a policy that maps from user states to sys- tem actions such that the overall expected cost of the dialogue is minimized. Such optimiza- tion typically requires data from dialogue corpora, which are manually annotated with task-oriented dialogue-act tags. On the other hand, the benefit of a hybrid approach is that human domain knowl- edge can be used to constrain the possible user states in the dialogue manager. We follow this idea by using an IDG, which expresses the task-domain knowledge through a directed graph of states from more general intention categories to more specific parameters of the intention categories. Figure 1 shows an example of such a graph, where each node is associated with a (potentially partial) user intention. In previous studies, this kind of do- main knowledge is used to restrict the user …

Content truncated to 500 words