Detecting Context Dependent Messages in a Conversational Environment

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1990–1999, Osaka, Japan, December 11-17 2016. Detecting Context Dependent Messages in a Conversational Environment Chaozhuo Li†, Yu Wu†, Wei Wu‡, Chen Xing⋄, Zhoujun Li†, Ming Zhou‡ †State Key Lab of Software Development Environment, Beihang University, Beijing, China ‡ Microsoft Research, Beijing, China ⋄ Nankai University, Tianjin, China {lichaozhuo,wuyu,lizj}@buaa.edu.cn {wuwei,v-chxing,mingzhou}@microsoft.com Abstract While automatic response generation for building chatbot systems has drawn a lot of attention recently, there is limited understanding on when we need to consider the linguistic context of an input text in the generation process. The task is challenging, as messages in a conversational environment are short and informal, and evidence that can indicate a message is context depen- dent is scarce. After a study of social conversation data crawled from the web, we observed that some characteristics estimated from the responses of messages are discriminative for identifying context dependent messages. With the characteristics as weak supervision, we propose using a Long Short Term Memory (LSTM) network to learn a classifier. Our method carries out text representation and classifier learning in a unified framework. Experimental results show that the proposed method can significantly outperform baseline methods on accuracy of classification. 1 Introduction Together with the rapid growth of social media such as Twitter and Weibo, the amount of conversation data on the web has tremendously increased. This makes building open domain chatbot systems with data-driven approaches possible. To carry on reasonable conversations with humans, a chatbot system needs to generate proper response with regard to users’ messages. Recently, with the large amount of conversation data available, learning a response generator from data has drawn a lot of attention (Ritter et al., 2011; Shang et al., 2015; Vinyals and Le, 2015). A key step to coherent response generation is determining when to consider linguistic context of mes- sages. Existing work on response generation, however, has overlooked this step. They either totally ignores linguistic context (Ritter et al., 2011; Shang et al., 2015; Vinyals and Le, 2015) or simply con- siders context for every message (Sordoni et al., 2015b; Serban et al., 2015). The former case is easy to lead to irrelevant responses when users’ input messages rely on the context information in previous con- versation turns, while the latter case is costly (e.g., on memory and responding time) for building a real chatbot system and has the risk of bringing in noise to response generation especially when users want to end the current conversation topic and start a new one. According to our observation, there are two types of messages in a conversational environment. The first type is context dependent message, which means to reply to the message, one must consider previous utterances in the dialogue1, while the second type is context independent message, which means even without the previous utterances, the message itself can still lead to a reasonable response. Table 1 compares the two types of messages using examples. In Case 1, “why do …

Content truncated to 500 words