Exploring Convolutional and Recurrent Neural Networks in Sequential Labelling for Dialogue Topic Tracking

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 963–973, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Exploring Convolutional and Recurrent Neural Networks in Sequential Labelling for Dialogue Topic Tracking Seokhwan Kim, Rafael E. Banchs, Haizhou Li Human Language Technology Department Institute for Infocomm Research Singapore 138632 {kims,rembanchs,hli}@i2r.a-star.edu.sg Abstract Dialogue topic tracking is a sequential la- belling problem of recognizing the topic state at each time step in given dialogue sequences. This paper presents various ar- tificial neural network models for dialogue topic tracking, including convolutional neural networks to account for semantics at each individual utterance, and recurrent neural networks to account for conversa- tional contexts along multiple turns in the dialogue history. The experimental results demonstrate that our proposed models can significantly improve the tracking perfor- mances in human-human conversations. 1 Introduction A human conversation often involves a series of multiple topics contextually related to each other. In this scenario, every participant in the conversa- tion is required to understand the on-going topic discussed at each moment, detect any topic shift made by others, and make a decision to self- initiate a new topic. These human capabilities for handling topics are also expected from dialogue systems to achieve natural and human-like conver- sations. Many studies have been conducted on multi- domain or multi-task dialogue systems by means of sentence-level topic identification as a sub- task of natural language understanding (Lin et al., 1999; Nakata et al., 2002; Lagus and Kuu- sisto, 2002; Adams and Martell, 2008; Ikeda et al., 2008; Celikyilmaz et al., 2011). In these ap- proaches, a given user input at each turn is cate- gorized into topic classes, each of which triggers the corresponding sub-system specializing in the particular topic. Despite many previous efforts, the sentence categorization methods still have the following limitations. Firstly, the effectiveness of the approaches is limited only in user-initiative conversations, because the categorization is per- formed mainly based on the user’s input men- tioned at a given turn. Secondly, no correlation be- tween different topics is considered neither in the topic decision process nor in each topic-specific sub-system operated independently from the oth- ers. Lastly, the conversational coherence in a given dialogue history sequence has limited effects on determining the current topic. Another direction for multi-topic dialogue sys- tems has been towards utilizing human knowledge represented in domain models (Roy and Subrama- niam, 2006) and agendas (Bohus and Rudnicky, 2003; Lee et al., 2008). The knowledge-based approaches make the system capable of having more control of dialogue flows including topic se- quences. This aspect contributes to better deci- sions of topics in system-initiative cases, but it can adversely affect the flexibility to deal with un- expected inputs against the system’s suggestions. Moreover, the high cost of building the required resources is another problem that these methods face from a practical point of view. Recently, some researchers (Morchid et al., 2014a; Morchid et al., 2014b; Esteve et al., 2015) have worked on topic identification for …

Content truncated to 500 words