How Can We Know What Language Models Know?

How Can We Know What Language Models Know? Zhengbao Jiang1∗ Frank F. Xu1∗ Jun Araki2 Graham Neubig1 1Language Technologies Institute, Carnegie Mellon University 2Bosch Research North America {zhengbaj,fangzhex,gneubig}@cs.cmu.edu jun.araki@us.bosch.com Abstract Recent work has presented intriguing results examining the knowledge contained in lan- guage models (LMs) by having the LM fill in the blanks of prompts such as ‘‘Obama is a by profession’’. These prompts are usually manually created, and quite possibly sub-optimal; another prompt such as ‘‘Obama worked as a ’’ may result in more accurately predicting the correct profession. Because of this, given an inappropriate prompt, we might fail to retrieve facts that the LM does know, and thus any given prompt only provides a lower bound estimate of the knowledge con- tained in an LM. In this paper, we attempt to more accurately estimate the knowledge con- tained in LMs by automatically discovering better prompts to use in this querying process. Specifically, we propose mining-based and paraphrasing-based methods to automatically generate high-quality and diverse prompts, as well as ensemble methods to combine answers from different prompts. Extensive experiments on the LAMA benchmark for extracting relational knowledge from LMs demonstrate that our methods can improve accuracy from 31.1% to 39.6%, providing a tighter lower bound on what LMs know. We have released the code and the resulting LM Prompt And Query Archive (LPAQA) at https://github.com/jzbjyb/LPAQA. 1 Introduction Recent years have seen the primary role of lan- guage models (LMs) transition from generating or evaluating the fluency of natural text (Mikolov and Zweig, 2012; Merity et al., 2018; Melis et al., 2018; Gamon et al., 2005) to being a powerful toolfor textunderstanding. This understanding has mainly been achieved through the use of language modeling as a pre-training task for feature extrac- tors, where the hidden vectors learned through a language modeling objective are then used in ∗ The first two authors contributed equally. down-stream language understanding systems (Dai and Le, 2015; Melamud et al., 2016; Peters et al., 2018; Devlin et al., 2019). Interestingly, it is also becoming apparent that LMs1 themselves can be used as a tool for text understanding by formulating queries in natural language and either generating textual answers directly (McCann et al., 2018; Radford et al., 2019), or assessing multiple choices and picking the most likely one (Zweig and Burges, 2011; Rajani et al., 2019). For example, LMs have been used to answer factoid questions (Radford et al., 2019), answer common sense queries (Trinh and Le, 2018; Sap et al., 2019), or extract factual knowledge about relations between entities (Petroni et al., 2019; Baldini Soares et al., 2019). Regardless of the end task, the knowledge contained in LMs is probed by providing a prompt, and letting the LM either generate the continuation of a prefix (e.g., ‘‘Barack Obama was born in ’’), or predict missing words in a cloze-style template (e.g., ‘‘Barack Obama is a by profession’’). However, while this paradigm has been used to achieve a number of intriguing results regarding the knowledge expressed by …

Content truncated to 500 words