When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 9802–9822 July 9-14, 2023 ©2023 Association for Computational Linguistics When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories Alex Mallen∗♢ Akari Asai∗♢ Victor Zhong♢ Rajarshi Das♢ Daniel Khashabi♠ Hannaneh Hajishirzi♢♡ ♢University of Washington ♠Johns Hopkins University ♡Allen Institute for AI {atmallen,akari,vzhong,rajarshi,hannaneh}@cs.washington.edu danielk@jhu.edu Abstract Despite their impressive performance on di- verse tasks, large language models (LMs) still struggle with tasks requiring rich world knowl- edge, implying the difficulty of encoding a wealth of world knowledge in their param- eters. This paper aims to understand LMs’ strengths and limitations in memorizing factual knowledge, by conducting large-scale knowl- edge probing experiments on two open-domain entity-centric QA datasets: POPQA, our new dataset with 14k questions about long-tail enti- ties, and EntityQuestions, a widely used open- domain QA dataset. We find that LMs struggle with less popular factual knowledge, and that retrieval augmentation helps significantly in these cases. Scaling, on the other hand, mainly improves memorization of popular knowledge, and fails to appreciably improve memorization of factual knowledge in the long tail. Based on those findings, we devise a new method for retrieval augmentation that improves per- formance and reduces inference costs by only retrieving non-parametric memories when nec- essary.1 1 Introduction Large language models (LMs; Brown et al. 2020; Raffel et al. 2020) have been shown to be compet- itive on diverse NLP tasks, including knowledge- intensive tasks that require fine-grained memoriza- tion of factual knowledge (Chowdhery et al., 2022; Yu et al., 2022). Meanwhile, LMs have also been shown to have limited memorization for less fre- quent entities (Kandpal et al., 2022), are prone to hallucinations (Shuster et al., 2021), and suf- fer from temporal degradation (Kasai et al., 2022; Jang et al., 2022). Incorporating non-parametric knowledge (i.e., retrieved text chunks) largely helps address those issues stemming from reliance on LMs’ parametric knowledge—knowledge stored 1Our code and data are available at https://github. com/AlexTMallen/adaptive-retrieval. Not memorized in parameters ⇨ use retrieval Memorized in parameters ⇨ don't use retrieval What is Kathy Saltzman's occupation? What is the capital of Louisiana? Figure 1: Relationship between subject entity popularity in a question and GPT-3 performance in open-domain QA, with and without retrieved passages. Adaptive Retrieval only retrieves when necessary (orange bars) based on the heuristically-decided threshold (red line). in their parameters (Izacard et al., 2022b)—but it is unclear whether it is strictly superior or comple- mentary to parametric knowledge. Understanding when we should not trust LMs’ outputs is also crucial to safely deploying them in real-world ap- plications (Kadavath et al., 2022). This work conducts a large-scale knowledge probing of LMs on factual knowledge memoriza- tion, to understand when we should and should not rely on LMs’ parametric knowledge, and how scal- ing and non-parametric memories (e.g., retrieval- augmented LMs) can help. In particular, we aim to address the following research questions: (RQ1) How much factual knowledge is memo- rized by LMs and what factors affect …

Content truncated to 500 words