Memory-assisted prompt editing to improve GPT-3 after deployment

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2833 - 2861 December 7-11, 2022 ©2022 Association for Computational Linguistics MemPrompt: Memory-assisted Prompt Editing with User Feedback Aman Madaan ∗, Niket Tandon ∗ †, Peter Clark†, Yiming Yang Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA, USA † Allen Institute for Artificial Intelligence, Seattle, WA, USA {amadaan,yiming}@cs.cmu.edu {nikett,peterc}@allenai.org Abstract Large LMs such as GPT-3 are powerful, but can commit mistakes that are obvious to hu- mans. For example, GPT-3 would mistakenly interpret "What word is similar to good?" to mean a homophone, while the user intended a synonym. Our goal is to effectively correct such errors via user interactions with the sys- tem but without retraining, which will be pro- hibitively costly. We pair GPT-3 with a grow- ing memory of recorded cases where the model misunderstood the user’s intents, along with user feedback for clarification. Such a mem- ory allows our system to produce enhanced prompts for any new query based on the user feedback for error correction on similar cases in the past. On four tasks (two lexical tasks, two advanced ethical reasoning tasks), we show how a (simulated) user can interactively teach a deployed GPT-3, substantially increasing its accuracy over the queries with different kinds of misunderstandings by the GPT-3. Our ap- proach is a step towards the low-cost utility enhancement for very large pre-trained LMs.1 1 Introduction Language models are now better than ever before at generating realistic content, but still lack common- sense (Bender and Koller, 2020; Marcus, 2021). One failure mode due to a lack of commonsense is in misunderstanding a user’s intent. The typical remedy of retraining with more data is prohibitive due to the cost and infrastructure requirements. In such cases, even if users repeatedly observe the model making a mistake, there are no avenues to provide feedback to the model to make it more accurate and personalized over time. Our goal is to allow users to correct such errors directly through interaction, and without retraining by injecting the knowledge required to correct the ∗Equal Contribution 1Code, data, and instructions to implement MemPrompt for a new task at https://www.memprompt.com/ Our memory enhanced GPT-3 implementation. User: What word is similar to good? GPT-3: The homophone of good is: wood. User: "Similar to" means "with similar meaning". GPT-3: Noted [writes to memory] User: What word is similar to surprised? GPT-3: The synonym of surprised is: amazed. [Retrieves and adds to prompt ‘"Similar to" means "with similar meaning"’]. Figure 1: This paper enhances GPT-3 performance by looking up questions with a similar intent that received any user feedback. Our approach is simple because only the question in the prompt needs to be updated with relevant feedback, and no retraining is necessary. model’s misunderstanding. Building upon the re- cent success of injecting commonsense in the input (Lewis et al., 2020; Talmor et al., 2020), we pro- pose a novel approach of injecting knowledge in the input via interactive feedback from an end-user. …

Content truncated to 500 words