A Hierarchical Neural Model for Learning Sequences of Dialogue Acts

Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 428–437, Valencia, Spain, April 3-7, 2017. c⃝2017 Association for Computational Linguistics A Hierarchical Neural Model for Learning Sequences of Dialogue Acts Quan Hung Tran and Ingrid Zukerman and Gholamreza Haffari Faculty of Information Technology, Monash University Clayton, VICTORIA 3800, Australia {hung.tran,ingrid.zukerman,gholamreza.haffari}@monash.edu Abstract We propose a novel hierarchical Recurrent Neural Network (RNN) for learning se- quences of Dialogue Acts (DAs). The in- put in this task is a sequence of utterances (i.e., conversational contributions) com- prising a sequence of tokens, and the out- put is a sequence of DA labels (one label per utterance). Our model leverages the hi- erarchical nature of dialogue data by using two nested RNNs that capture long-range dependencies at the dialogue level and the utterance level. This model is combined with an attention mechanism that focuses on salient tokens in utterances. Our exper- imental results show that our model out- performs strong baselines on two popular datasets, Switchboard and MapTask; and our detailed empirical analysis highlights the impact of each aspect of our model. 1 Introduction The sequence-labeling task involves learning a model that maps an input sequence to an output sequence. Many NLP problems can be treated as sequence-labeling tasks, e.g., part-of-speech (PoS) tagging (Toutanova et al., 2003; Toutanova and Manning, 2000), machine translation (Brown et al., 1993) and automatic speech recognition (Gales and Young, 2008). Recurrent Neural Nets (RNNs) have been the workhorse model for many NLP sequence-labeling tasks, e.g., machine transla- tion (Sutskever et al., 2014) and speech recogni- tion (Amodei et al., 2015), due to their ability to capture long-range dependencies inherent in natu- ral language. In this paper, we propose a hierarchical RNN for labeling a sequence of utterances (i.e., con- tributions) in a dialogue with their Dialogue Acts (DAs). This task is particularly useful for dialogue systems, as knowing the DA of an utterance sup- ports its interpretation, and the generation of an appropriate response. The DA classification prob- lem differs from the aforementioned tasks in the structure of the input and the immediacy of the output. The input in these tasks is a sequence of tokens, e.g., a sequence of words in PoS tagging; while in DA classification, the input is hierarchi- cal, i.e., a conversation comprises a sequence of utterances, each of which has a sequence of to- kens (Figure 1). In addition, to be useful for dia- logue systems, the DA of an utterance must be de- termined immediately, hence a bi-directional ap- proach is not feasible. As mentioned above, RNNs are able to capture long-range dependencies. This ability was har- nessed by Shen and Lee (2016) for DA classifica- tion. However, they ignored the conversational di- mension of the data, treating the utterances in a di- alogue as separate instances — an assumption that results in loss of information. To overcome this limitation, we designed a two-layer RNN model that leverages the hierarchical nature of dialogue …

Content truncated to 500 words