Chat or Learn: a Data-Driven Robust Question-Answering System

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 5474–5480 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 5474 Chat or Learn: a Data-Driven Robust Question-Answering System Gabriel Luthier and Andrei Popescu-Belis HEIG-VD / HES-SO Route de Cheseaux 1, CP 521 1401 Yverdon-les-Bains, Switzerland {gabriel.luthier, andrei.popescu-belis}@heig-vd.ch Abstract We present a voice-based conversational agent which combines the robustness of chatbots and the utility of question answering (QA) systems. Indeed, while data-driven chatbots are typically user-friendly but not goal-oriented, QA systems tend to perform poorly at chitchat. The proposed chatbot relies on a controller which performs dialogue act classification and feeds user input either to a sequence-to-sequence chatbot or to a QA system. The resulting chatbot is a spoken QA application for the Google Home smart speaker. The system is endowed with general-domain knowledge from Wikipedia articles and uses coreference resolution to detect relatedness between questions. We present our choices of data sets for training and testing the components, and present the experimental results that helped us optimize the parameters of the chatbot. In particular, we discuss the appropriateness of using the SQuAD dataset for evaluating end-to-end QA, in the light of our system’s behavior. Keywords: question answering, dialogue acts, chatbots, SQuAD, Wikipedia, movie subtitles. 1. Introduction The multiplication of voice-based virtual assistants avail- able on mobile phones or smart speakers has made conver- sational systems familiar to the general public. Commer- cial assistants from large technology companies focused initially on tasks related to the main offerings of these companies: controlling a smartphone (Apple’s Siri), pur- chasing goods or services (Amazon’s Alexa), or searching for information over the Web (Google Assistant). Beyond these functionalities, which can in principle be achieved with large finite-state dialogue models, commercial assis- tants were not intended to support extended conversations on topics in the general domain, although their capacities for chitchat have been gradually increasing (Fang et al., 2018). On the contrary, the systems known as chatbots were intended from the start to offer robust conversational ca- pacities, although with a thin and often implicit knowledge base. Such chatbots are often trained on large dialogue cor- pora from movies or social media. However, giving them task-oriented capabilities is difficult, at least when using the sequence-to-sequence neural models on which most recent chatbots rely. In this paper, we present a simple yet effective solution to design a conversational agent that combines the robust- ness of chatbots with the utility of task-oriented systems – here, question answering (QA). The system supports task- oriented dialogues, to answer factual questions over a doc- ument set such as Wikipedia, but has also the capacity to reply to non-task-oriented questions, for managing the so- cial aspects of the conversation including chitchat (or small talk). Ahead of the two components supporting these func- tionalities, which are implemented using state-of-the-art technologies, the system relies on a controller, which per- forms dialogue act recognition and routes user utterances to one of the components. The system operates using …

Content truncated to 500 words