Dialogue-act-driven Conversation Model : An Experimental Study

Proceedings of the 27th International Conference on Computational Linguistics, pages 1246–1256 Santa Fe, New Mexico, USA, August 20-26, 2018. 1246 Dialogue-act-driven Conversation Model: An Experimental Study Harshit Kumar IBM Research, Delhi, India harshitk@in.ibm.com Arvind Agarwal IBM Research, Delhi, India arvagarw@in.ibm.com Sachindra Joshi IBM Research, Delhi, India jsachind@in.ibm.com Abstract The utility of additional semantic information for the task of next utterance selection in an auto- mated dialogue system is the focus of study in this paper. In particular, we show that additional information available in the form of dialogue acts –when used along with context given in the form of dialogue history– improves the performance irrespective of the underlying model being generative or discriminative. In order to show the model agnostic behavior of dialogue acts, we experiment with several well-known models such as sequence-to-sequence encoder-decoder model, hierarchical encoder-decoder model, and Siamese-based models with and without hier- archy; and show that in all models, incorporating dialogue acts improves the performance by a significant margin. We, furthermore, propose a novel way of encoding dialogue act information, and use it along with hierarchical encoder to build a model that can use the sequential dialogue act information in a natural way. Our proposed model achieves an MRR of about 84.8% for the task of next utterance selection on a newly introduced DailyDialog dataset, and outperform the baseline models. We also provide a detailed analysis of results including key insights that explain the improvement in MRR because of dialogue act information. 1 Introduction In the last decade, natural language processing and machine learning –in particular deep learning– have come a long way towards building an automated dialogue system. In a fully automated dialogue sys- tem, the goal is to predict an appropriate response given the dialogue history. This problem of response prediction can be formulated in two ways. One is purely generative, where the task is to generate a text response, i.e. generating a sentence or utterance from scratch, whereas the other is Next Utterance Selection, where the task is to select an appropriate response from a set of given candidates. Despite significant research in text generation, a pure generative model capable of generating syntactically and semantically correct text still remains a distant reality. There have been several efforts such as (Vinyals and Le, 2015; Serban et al., 2016a; Serban et al., 2016b; Serban et al., 2017b) for the task of dialogue generation, however these models still do not seem to work in practice (Liu et al., 2016). This is partic- ularly true for open domain dialogue systems. Dialogue generation in a task-oriented oriented dialogue system, such as flight-booking and troubleshooting, is much easier than in a non-task oriented dialogue system. This level of difficulty arises because a non-task-oriented dialogue system has no predefined goal (or domain), and the vocabulary and possibilities of the dialogues could be endless. Given these challenges, researchers have defined a simpler problem for conversation modeling based on retrieval, i.e. next utterance selection. In this paper we use this second formulation of …

Content truncated to 500 words