Towards Teachable Reasoning Systems: Using a Dynamic Memory of User Feedback for Continual System Improvement

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9465 - 9480 December 7-11, 2022 ©2022 Association for Computational Linguistics Towards Teachable Reasoning Systems: Using a Dynamic Memory of User Feedback for Continual System Improvement Bhavana Dalvi Mishra, Oyvind Tafjord, Peter Clark Allen Institute for AI, Seattle, WA {bhavanad,oyvindt,peterc}@allenai.org Abstract Our goal is a teachable reasoning system for question-answering (QA), where a user can in- teract with faithful answer explanations, and correct its errors so that the system improves over time. Our approach is to augment a QA model with a dynamic memory of user feed- back, containing user-supplied corrections to erroneous model beliefs that users identify dur- ing interaction. Retrievals from memory are used as additional context for QA, to help avoid previous mistakes in similar new situations - a novel application of memory-based contin- uous learning. With simulated feedback, we find that our system (called TeachMe1) contin- ually improves with time, and without model retraining, requiring feedback on only 25% of training examples to reach within 1% of the upper-bound (feedback on all examples). Sim- ilarly, in experiments with real users, we ob- serve a similar trend, with performance improv- ing by over 15% on a hidden test set after teach- ing. This suggests new opportunities for using frozen language models in an interactive setting where users can inspect, debug, and correct the model’s beliefs, leading to improved system’s performance over time. 1 Introduction Our goal is a teachable question-answering (QA) system - one that a user can interact with to see faithful explanations for its answers, debug errors, and correct them so that the system gradually im- proves over time (sometimes referred to as explana- tory interactive machine learning (XIL) (Teso and Kersting, 2019)). While the benefits of such a sys- tem are evident (Lakkaraju et al., 2022), the chal- lenges are evident also: despite recent progress in explainability (Wiegreffe and Marasovi´c, 2021), it is often hard to understand how a model arrived at an answer, and even harder to correct it if it made 1Supplementary data and models are available at https: //allenai.org/data/teachme Figure 1: TeachMe augments the basic question- answering model with a memory of user feedback. (A) Given a new question, facts retrieved from memory are used as additional context for the model, influencing its answers and proofs. (B) If the user disagrees with an answer, they localize the error in the explanation and offer corrective feedback, which is added to memory. (C) These new facts can then be retrieved if the query is re-asked, helping the system avoid repeating mistakes. Note that these also help improve answers on new, sim- ilar questions that are asked later, helping the system improve over time. a mistake. In contrast, people are typically able to provide a chain of reasoning for their decisions, and may change their mind if a flaw in their knowledge or reasoning is exposed. Our goal is to similarly have machines provide reasoned answers to ques- tions, showing how …

Content truncated to 500 words