BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8849–8861 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 8849 BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief Nora Kassner1,2, Oyvind Tafjord1, Hinrich Schütze2, Peter Clark1 1Allen Institute for AI, Seattle, WA 2Center for Information and Language Processing, LMU Munich, Germany kassner@cis.lmu.de {oyvindt,peterc}@allenai.org Abstract Although pretrained language models (PTLMs) contain significant amounts of world knowledge, they can still produce inconsistent answers to questions when probed, even after specialized training. As a result, it can be hard to identify what the model actually “believes” about the world, making it sus- ceptible to inconsistent behavior and simple errors. Our goal is to reduce these problems. Our approach is to embed a PTLM in a broader system that also includes an evolving, symbolic memory of beliefs – a BeliefBank – that records but then may modify the raw PTLM answers. We describe two mechanisms to improve belief consistency in the overall system. First, a reasoning component – a weighted MaxSAT solver – revises beliefs that significantly clash with others. Second, a feedback component issues future queries to the PTLM using known beliefs as context. We show that, in a controlled experimental setting, these two mechanisms result in more consis- tent beliefs in the overall system, improving both the accuracy and consistency of its answers over time. This is significant as it is a first step towards PTLM-based architectures with a systematic notion of belief, enabling them to construct a more coherent picture of the world, and improve over time without model retraining. 1 Introduction Intelligent agents are typically considered to have beliefs about the world – propositions that they take as true (Genin and Huber, 2021). In general, a sys- tem can be said to (appear to) believe a proposition p, e.g., “eagles are birds”, if it produces answers consistent with p (and its other beliefs). Pragmati- cally, we expect the system to (a) give a consistent answer to different paraphrases of the question "p?" ("Are eagles birds?", "Is an eagle a type of bird?", ...), and (b) give correct answers about implications of p ("Eagles lay eggs", "Eagles have feathers", ...), Figure 1: The proposed architecture. The model’s raw answers are stored in a persistent, symbolic memory (BeliefBank), and two mechanisms attempt to improve them: (a) A constraint solver flips beliefs (e.g., the be- lief that “a swallow is a fish”) that clash significantly with others. (b) A feedback mechanism poses new questions using existing, relevant beliefs (e.g., “a swal- low is not a fish”) as the query context. We find that both consistency and accuracy of the overall system improve. Example: The model M shown in the fig- ure incorrectly answers “yes”, when asked “a swallow has gills?”. But (as shown above) if reminded of its previous answer “a swallow is not a fish”, M correctly answers "no". conditional on its other knowledge and reasoning abilities. Maintaining a consistent set of beliefs …

Content truncated to 500 words