SimVecs: Similarity-Based Vectors for Utterance Representation in Conversational AI Systems

Proceedings of the 23rd Conference on Computational Natural Language Learning, pages 708–717 Hong Kong, China, November 3-4, 2019. c⃝2019 Association for Computational Linguistics 708 SIMVECS: Similarity-based Vectors for Utterance Representation in Conversational AI Systems Ashraf Mahgoub Purdue University / West Lafayette, IN amahgoub@purdue.edu Riham Mansour Microsoft / Redmond, WA rihamma@microsoft.com Youssef Shahin Microsoft / Redmond, WA Youssef.Shahin@microsoft.com Saurabh Bagchi Purdue University / West Lafayette, IN sbagchi@purdue.edu Abstract Conversational AI systems are gaining a lot of attention recently in both industrial and scientific domains, providing a natural way of interaction between customers and adap- tive intelligent systems. A key requirement in these systems is the ability to efficiently parse user queries, understand the intent be- hind each query, and provide adequate re- sponses to users. Therefore, many applications such as conversation bots and smart IoT de- vices has a natural language understanding (LU) service integrated within. One of the greatest challenges of language understanding services is efficient utterance (sentence) repre- sentation in vector space, which is an essential step for most ML tasks. In this paper, we pro- pose a novel approach for generating vector space representations of conversational utter- ances using pair-wise similarity metrics. The proposed approach uses only a few corpora to tune the weights of the similarity metric with- out relying on external general purpose ontolo- gies. Our experiments confirm that the gener- ated vectors can improve the performance of LU services in unsupervised, semi-supervised and supervised learning tasks over state-of- the-art prior works. 1 Introduction Challenges of conversational AI systems: Con- versational AI systems empower virtual assistants in many applications such as conversation bots (also known as Chatbots) and smart IoT devices. Their ability to understand user spoken commands and identify the user’s intent(s) is one of their main benefits. However, this is a very challeng- ing task due to the diversity of domains and lan- guages they are required to support. For example, Microsoft LUIS1 currently supports 12 languages, 1www.luis.ai/home whereas IBM Watson 2 supports 10 languages in their language understanding services. Moreover, conversational text queries are often very short and sparse, which hinders conventional text represen- tations such as Bag-of-words (BOW) from captur- ing adequate features of the utterance. In LU services, a language understanding ap- plication is often represented as a group of intents and entities that serve a specific business applica- tion. For example, a restaurant application may de- fine intents such as Order Food, Show Menu and Cancel Order. The task of the language under- standing service then is to classify newly received queries (utterances) into one (or more) of the de- fined intents. Although many prior works were focused on constructing word-level vector representation such as (Mikolov et al., 2013) and (Pennington et al., 2014), generating an utterance-level vector repre- sentation is still a challenging task. Existing lan- guage modeling (LM) based approaches such as Para2Vec (Le and Mikolov, 2014) rely on deep neural networks to generate vector representa- tions for the paragraph-level. However, these ap- proaches are normally trained with …

Content truncated to 500 words