Modeling Multi-Action Policy for Task-Oriented Dialogues

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1304–1310, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1304 Modeling Multi-Action Policy for Task-Oriented Dialogues Lei Shu, Hu Xu, Bing Liu University of Illinois at Chicago {lshu3, hxu48, liub}@uic.edu Piero Molino Uber AI piero@uber.com Abstract Dialogue management (DM) plays a key role in the quality of the interaction with the user in a task-oriented dialogue system. In most existing approaches, the agent predicts only one DM policy action per turn. This sig- nificantly limits the expressive power of the conversational agent and introduces unwanted turns of interactions that may challenge users’ patience. Longer conversations also lead to more errors and the system needs to be more robust to handle them. In this paper, we com- pare the performance of several models on the task of predicting multiple acts for each turn. A novel policy model is proposed based on a recurrent cell called gated Continue-Act-Slots (gCAS) that overcomes the limitations of the existing models. Experimental results show that gCAS1 outperforms other approaches. 1 Introduction In a task-oriented dialogue system, the dialogue manager policy module predicts actions usually in terms of dialogue acts and domain specific slots. It is a crucial component that influences the ef- ficiency (e.g., the conciseness and smoothness) of the communication between the user and the agent. Both supervised learning (SL) (Stent, 2002; Williams et al., 2017a; Williams and Zweig, 2016; Henderson et al., 2005, 2008) and reinforcement learning (RL) approaches (Walker, 2000; Young et al., 2007; Gasic and Young, 2014; Williams et al., 2017b; Su et al., 2017) have been adopted to learn policies. SL learns a policy to predict acts given the dialogue state. Recent work (Wen et al., 2017; Liu and Lane, 2018) also used SL as pre-training for RL to mitigate the sam- ple inefficiency of RL approaches and to reduce the number of interactions. Sequence2Sequence 1The code is available at https://leishu02. github.io/ user msg Hi! I’m looking for good thriller. Are there any playing right now? agent msg Yes, there are! The Witch, The Other Side of the Door, and The Boy are all thrillers. Would you like to find tickets for a showing for any of them? agent acts inform(moviename=The Witch, The Other Side of the Door, The Boy; genre=thriller) multiple choice(moviename) Table 1: Dialogue example. (Seq2Seq) (Sutskever et al., 2014) approaches have also been adopted in user simulators to pro- duce user acts (Gur et al., 2018). These ap- proaches typically assume that the agent can only produce one act per turn through classification. Generating only one act per turn significantly lim- its what an agent can do in a turn and leads to lengthy dialogues, making tracking of state and context throughout the dialogue harder. An exam- ple in Table 1 shows how the agent can produce both an inform and a multiple choice act, reducing the need for additional turns. The …

Content truncated to 500 words