Crawling The Internal Knowledge-Base of Language Models

Findings of the Association for Computational Linguistics: EACL 2023, pages 1856–1869 May 2-6, 2023 ©2023 Association for Computational Linguistics Crawling The Internal Knowledge-Base of Language Models Roi Cohen1 Mor Geva2∗ Jonathan Berant1 Amir Globerson1 1Blavatnik School of Computer Science, Tel Aviv University 2Allen Institute for AI roi1@mail.tau.ac.il, pipek@google.com, joberant@cs.tau.ac.il, gamir@tauex.tau.ac.il Abstract Language models are trained on large volumes of text, and as a result their parameters might contain a significant body of factual knowledge. Any downstream task performed by these mod- els implicitly builds on these facts, and thus it is highly desirable to have means for represent- ing this body of knowledge in an interpretable way. However, there is currently no mechanism for such a representation. Here, we propose to address this goal by extracting a knowledge- graph of facts from a given language model. We describe a procedure for “crawling” the internal knowledge-base of a language model. Specifically, given a seed entity, we expand a knowledge-graph around it. The crawling pro- cedure is decomposed into sub-tasks, realized through specially designed prompts that con- trol for both precision (i.e., that no wrong facts are generated) and recall (i.e., the number of facts generated). We evaluate our approach on graphs crawled starting from dozens of seed en- tities, and show it yields high precision graphs (82-92%), while emitting a reasonable number of facts per entity. 1 Introduction Modern language models (LMs) (Raffel et al., 2020; Brown et al., 2020) are trained on vast amounts of text that captures much of human knowledge, including scientific articles, Wikipedia, books, and other sources of information (Gao et al., 2020). Consequently, such models encode world knowledge in their parameters, allowing them to generate rich and coherent outputs. Past work has illustrated LMs can be viewed as knowledge-bases (Petroni et al., 2019) as well as analyzed the encoded knowledge (e.g., see AlKhamissi et al., 2022) and leveraged it for applications such as closed-book QA (Roberts et al., 2020; Brown et al., 2020) and search (Tay et al., 2022), illustrating LMs can be viewed as ∗ Now at Google Research. knowledge-bases (Petroni et al., 2019). But what are the facts stored in the internal knowledge bases of modern LMs, and how can these be represented explicitly? This is the challenge we address in this work. Our motivation is to obtain an interpretable and transparent representation that will allow hu- mans to inspect what the LM knows, what it does not know, why it makes certain mistakes, and what are the biases it encodes. Moreover, with such a representation, one can leverage general-purpose tools, such as query languages, for interacting with this knowledge. The first question in this endeavour is what is a suitable explicit knowledge representation. A natural candidate structure is a knowledge graph (KG). Namely, a graph whose nodes are entities and whose edges represent relations between enti- ties. KGs are appealing since information can be readily “read-off” from the graph, they can be reli- ably queried, and different KGs can be easily com- pared. …

Content truncated to 500 words