Classifying Dialogue Acts in One-on-One Live Chats

Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 862–871, MIT, Massachusetts, USA, 9-11 October 2010. c⃝2010 Association for Computational Linguistics Classifying Dialogue Acts in One-on-one Live Chats Su Nam Kim,♠ Lawrence Cavedon♥ and Timothy Baldwin♠ ♠ Dept of Computer Science and Software Engineering, University of Melbourne ♥ School of Computer Science and IT, RMIT University sunamkim@gmail.com, lcavedon@gmail.com, tb@ldwin.net Abstract We explore the task of automatically classify- ing dialogue acts in 1-on-1 online chat forums, an increasingly popular means of providing customer service. In particular, we investi- gate the effectiveness of various features and machine learners for this task. While a sim- ple bag-of-words approach provides a solid baseline, we find that adding information from dialogue structure and inter-utterance depen- dency provides some increase in performance; learners that account for sequential dependen- cies (CRFs) show the best performance. We report our results from testing using a corpus of chat dialogues derived from online shop- ping customer-feedback data. 1 Introduction Recently, live chats have received attention due to the growing popularity of chat services and the in- creasing body of applications. For example, large organizations are increasingly providing support or information services through live chat. One advan- tage of chat-based customer service over conven- tional telephone-based customer service is that it becomes possible to semi-automate aspects of the interaction (e.g. conventional openings or canned responses to standard questions) without the cus- tomer being aware of it taking place, something that is not possible with speech-based dialogue systems (as synthesised speech is still easily distinguishable from natural speech). Potentially huge savings can be made by organisations providing customer help services if we can increase the degree of automation of live chat. Given the increasing impact of live chat services, there is surprisingly little published computational linguistic research on the topic. There has been sub- stantially more work done on dialogue and dialogue corpora, mostly in spoken dialogue (e.g. Stolcke et al. (2000)) but also multimodal dialogue systems in application areas such as telephone support service (Bangalore et al., 2006) and tutoring systems (Lit- man and Silliman, 2004). Spoken dialogue analysis introduces many complications related to the error inherent in current speech recognition technologies. As an instance of written dialogue, an advantage of live chats is that recognition errors are not such an is- sue, although the nature of language used in chat is typically ill-formed and turn-taking is complicated by the semi-asynchronous nature of the interaction (e.g. Werry (1996)). In this paper, we investigate the task of automatic classification of dialogue acts in 1-on-1 live chats, focusing on “information delivery” chats since these are proving increasingly popular as part of enter- prise customer-service solutions. Our main chal- lenge is to develop effective features and classifiers for classifying aspects of 1-on-1 live chat. Much of the work on analysing dialogue acts in spoken di- alogues has relied on non-lexical features, such as prosody and acoustic features (Stolcke et al., 2000; Julia and Iftekharuddin, 2008; Sridhar et al., 2009), which …

Content truncated to 500 words