Classifying Dialogue Acts in Multi-party Live Chats

Copyright 2012 by Su Nam Kim, Lawrence Cavedon, and Timothy Baldwin 26th Pacific Asia Conference on Language,Information and Computation pages 463–472 Classifying Dialogue Acts in Multi-party Live Chats Su Nam Kim School of IT Monash University Clayton, VIC, Australia su.kim@monash.edu.au Lawrence Cavedon School of CS and IT RMIT University Melbourne, VIC, Australia lawrence.cavedon@rmit.edu.au Timothy Baldwin Dept. of Computing and Information Systems The University of Melbourne VIC, Australia tb@ldwin.net Abstract We consider the task of classifying chat con- tributions by dialogue act in a multi-party set- ting. This extends the problem significantly over the 1-1 chat scenario due to the semi- asynchronous and “entangled” nature of the contributions by chat participants. We ex- periment with a number of machine learning approaches, using different categories of fea- tures: lexical, contextual, structural, keyword and dialogue interaction information. For evaluation, we developed gold-standard data using online forums from the USA Library of Congress. We found that, for multi-party di- alogues, features based on 1-gram and key- words produced best performance, while fea- tures exploiting structure and interaction did not perform as well as previously reported re- sults over 1-to-1 chats. 1 Introduction Dialogue Acts (or DAs) are discourse units (or ut- terances) that represent the semantics of contribu- tions to a dialogue at the level of illocutionary force. Dialogue acts have been studied in various types of conversations — spoken/written dialogue contribu- tions (Stolcke et al., 2000; Wu et al., 2002; Kim et al., 2010a), sentence-level (Lampert et al., 2008), paragraph-level (Cong et al., 2008), or complete messages consisting of several paragraphs (Cohen et al., 2004). Authors have argued that automatic dia- logue act identification could help in a range of ap- plications, such as meeting summarisation (Murray et al., 2006), email summarisation, conversational agents, speech recognition (Stolcke et al., 2000), or human social intention detection (Jurafsky et al., 2009). They can also be useful in information- sharing chats in online forums (Kim et al., 2010b; Wang et al., 2011). Recently, live chat has received growing atten- tion since chat services and similar applications have gained popularity as a communication method. However, the majority of previous work on dialogue act classification for dialogue has been carried out over spoken dialogue. Although spoken and writ- ten dialogue have similarities, they have distinct fea- tures which make it difficult to reuse existing meth- ods for live chats. For example, spoken dialogue in- troduces difficulties due to errors inherent in speech recognition output, but allows acoustic and prosodic features to be leveraged (e.g. Stolcke et al. (2000)). Conversely, live chats introduce other types of com- plications, including ill-formed data and entangle- ment (especially for multi-party conversations) due to the semi-asynchronous nature of the interaction (e.g. (Werry, 1996)). As a result, studying live chats is a necessary step toward building accurate live chat systems. To date, relatively little work has targeted dia- logue act classification over live chat data. Wu et al. (2002) and Forsyth (2007) investigated multi-party casual chats, while Ivanovic (2008) and Kim et al. (2010a) …

Content truncated to 500 words