Learning End-to-End Goal-Oriented Dialog with Maximal User Task Success and Minimal Human Agent Use

Learning End-to-End Goal-Oriented Dialog with Maximal User Task Success and Minimal Human Agent Use Janarthanan Rajendran†∗, Jatin Ganhotra‡, and Lazaros C. Polymenakos‡ †University of Michigan ‡IBM Research rjana@umich.edu, jatinganhotra@us.ibm.com, and lcpolyme@us.ibm.com Abstract Neural end-to-end goal-oriented dialog sys- tems showed promise to reduce the workload of human agents for customer service, as well as reduce wait time for users. However, their inability to handle new user behavior at deployment has limited their usage in real world. In this work, we propose an end-to- end trainable method for neural goal-oriented dialog systems that handles new user behaviors at deployment by transferring the dialog to a human agent intelligently. The proposed method has three goals: 1) maximize user’s task success by transferring to human agents, 2) minimize the load on the human agents by transferring to them only when it is essential, and 3) learn online from the human agent’s responses to reduce human agents’ load fur- ther. We evaluate our proposed method on a modified-bAbI dialog task,1 which simulates the scenario of new user behaviors occurring at test time. Experimental results show that our proposed method is effective in achieving the desired goals. 1 Introduction Neural end-to-end dialog systems showed huge potential for various goal-oriented dialog tasks such as restaurant reservation, flight ticket booking, and hotel reservation. However, their use in the real world has been limited because of their inability to handle new user behavior at deployment. There are two main methods to build neural end-to-end goal-oriented dialog systems. In the ∗This work was done when the author was an intern at IBM Research, NY. 1modified-bAbI dialog tasks - https://github. com/IBM/modified-bAbI-dialog-tasks. first method, large amounts of human–human chat logs of a particular task are collected and then the dialog system is trained to mimic the chat logs using Supervised Learning (SL) (Bordes et al., 2017). In the second method, the dialog system is trained to complete the task against a human (user) simulator (Young et al., 2013). The training is done using Reinforcement Learning (RL) by providing reward for task completion and also intermediate rewards for pre-identified subtask completion. This is often accompanied by a SL pre-training, as in Liu and Lane (2017). Dialog systems built using either of these methods would fail in the presence of new user behaviors during deployment, which were missing in training. Failure here refers to the inability of the dialog system to complete the task for the user. The new behaviors can be a different way of a user asking/providing certain information or could also be as simple as an user utterance with out- of-vocabulary words. The failure happens when these new behaviors are beyond the generalization capabilities of the trained systems due to the limited coverage of training data collected. For a real-world use-case, it is difficult to collect chat logs and/or to build a user simulator that covers all possible user behaviors, which implies that users with new behaviors are bound to come by during deployment. The new user behaviors that we refer …

Content truncated to 500 words