Improving Conversational Question Answering Systems after Deployment using Feedback-Weighted Learning

Proceedings of the 28th International Conference on Computational Linguistics, pages 2561–2571 Barcelona, Spain (Online), December 8-13, 2020 2561 Improving Conversational Question Answering Systems after Deployment using Feedback-Weighted Learning Jon Ander Campos1, Kyunghyun Cho2, Arantxa Otegi1, Aitor Soroa1, Gorka Azkune1, Eneko Agirre1 1University of the Basque Country (UPV/EHU) 2New York University (NYU) 1 {jonander.campos, arantza.otegi, a.soroa, gorka.azkune, e.agirre}@ehu.eus, 2kyunghyun.cho@nyu.edu Abstract The interaction of conversational systems with users poses an exciting opportunity for improving them after deployment, but little evidence has been provided of its feasibility. In most applica- tions, users are not able to provide the correct answer to the system, but they are able to pro- vide binary (correct, incorrect) feedback. In this paper we propose feedback-weighted learning based on importance sampling to improve upon an initial supervised system using binary user feedback. We perform simulated experiments on document classification (for development) and Conversational Question Answering datasets like QuAC and DoQA, where binary user feedback is derived from gold annotations. The results show that our method is able to improve over the initial supervised system, getting close to a fully-supervised system that has access to the same labeled examples in in-domain experiments (QuAC), and even matching in out-of-domain exper- iments (DoQA). Our work opens the prospect to exploit interactions with real users and improve conversational systems after deployment. 1 Introduction In Conversational Question Answering (CQA) systems, the user makes a set of interrelated questions to the system, which extracts the answers from reference text (Choi et al., 2018). These systems are trained on datasets of human-human dialogues collected using Wizard-of-Oz techniques, where two crowd- sourcers are paired at random to emulate the questioner and the answerer. Several projects have shown that it is possible to train effective systems using such datasets. For instance, QuAC includes question and answers about popular people in Wikipedia (Choi et al., 2018), and DoQA includes question-answer conversations on cooking, movies and travel FAQs (Campos et al., 2020). Building such datasets comes at a cost, which limits the widespread use of conversational systems built using supervised learning. The fact that conversational systems interact naturally with users poses an exciting opportunity to im- prove them after deployment. Given enough training data, a company can deploy a basic conversational system, enough to be accepted and used by users. Once the system is deployed, the interaction with users and their feedback can be used to improve the system. In this work we focus on the case where a CQA system trained off-line is deployed and receives explicit binary (correct, incorrect) feedback from users. An example of this task can be seen in Figure 1 where at a point in the conversation two different users give binary feedback to the system according to the correctness of the received answer. Assuming a large number of interactions, we can safely ignore examples for which no feedback is received. We propose feedback-weighted learning (FWL) based on importance sampling as the technique to improve the initial supervised system using only binary feedback from users. …

Content truncated to 500 words