Intent Mining from past conversations for Conversational Agent

Proceedings of the 28th International Conference on Computational Linguistics, pages 4140–4152 Barcelona, Spain (Online), December 8-13, 2020 4140 Intent Mining from past conversations for Conversational Agent Ajay Chatterjee Accenture Labs, India ajay.chatt03@gmail.com Shubhashis Sengupta Accenture Labs, India shubhashis.sengupta@accenture.com Abstract Conversational systems are of primary interest in the AI community. Organizations are increas- ingly using chatbot to provide round-the-clock support and to increase customer engagement. Many commercial bot building frameworks follow a standard approach that requires one to build and train an intent model to recognize user input. These frameworks require a collection of user utterances and corresponding intent to train an intent model. Collecting a substantial coverage of training data is a bottleneck in the bot building process. In cases where past conversation data is available, the cost of labeling hundreds of utterances with intent labels is time-consuming and laborious. In this paper, we present an intent discovery framework that can mine a vast amount of conversational logs and to generate labeled data sets for training intent models. We have introduced an extension to the DBSCAN (Ester et al., 1996) algorithm and presented a density-based clustering algorithm ITER-DBSCAN for unbalanced data clustering. Empirical evaluation on one conversation dataset, six intent dataset, and one short text clustering dataset show the effectiveness of our hypothesis. We release the datasets and code for future evaluation at https://github.com/ajaychatterjee/IntentMining . 1 1 Introduction In the past few years, there is a growing community and business interest in conversational systems (chatbots primarily). A key step towards designing a task-oriented conversational model is to identify and understand the intention of a user utterance. An intent in a conversational model maps semantically similar sentences to a high-level abstraction for a chatbot that can generate a similar response or perform an action. For example, “unable to log-in to the system”, “can not log in”, “facing issue during sign-in” - though linguistically different, are all interpreted as intents related to login issue. The current crop of bot-building frameworks require annotated data for building an Intent model. Many commercial chatbot building frameworks such as Microsoft Azure Bot Service2, IBM Watson Assistant3 support intent train- ing in a supervised setting. The developers and domain experts typically consider past interaction logs between human-human or human-computer as a valuable resource and carry out an extensive manual process of intent labeling. The process of intent discovery and training data creation is by large manual and effort-intensive and carried out by domain experts. Existing dialog corpora contains pre-defined intent and dialog state defined, and consequently, most of the work (Mrkˇsi´c et al., 2015; Henderson et al., 2014) ignores intent discovery during conversation design. Previous work (Haponchyk et al., 2018; P, 2016) on intent identification focuses on clustering single user query/ question using supervised or unsupervised clustering. But the tasks do not consider conversational data. Perkins (2019) discusses the realistic complexity of user intent space in a complex domain such as customer support and health care and use the conversational data for clustering and intent induction. But …

Content truncated to 500 words