Can Prompt Probe Pretrained Language Models? Understanding the Invisible Risks from a Causal View

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 5796 - 5808 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Can Prompt Probe Pretrained Language Models? Understanding the Invisible Risks from a Causal View Boxi Cao1,3, Hongyu Lin1, Xianpei Han1,2,4∗, Fangchao Liu1,3, Le Sun1,2∗ 1Chinese Information Processing Laboratory 2State Key Laboratory of Computer Science Institute of Software, Chinese Academy of Sciences, Beijing, China 3University of Chinese Academy of Sciences, Beijing, China 4 Beijing Academy of Artificial Intelligence, Beijing, China {boxi2020,hongyu,xianpei,fangchao2017,sunle}@iscas.ac.cn Abstract Prompt-based probing has been widely used in evaluating the abilities of pretrained language models (PLMs). Unfortunately, recent stud- ies have discovered such an evaluation may be inaccurate, inconsistent and unreliable. Fur- thermore, the lack of understanding its inner workings, combined with its wide applicability, has the potential to lead to unforeseen risks for evaluating and applying PLMs in real-world applications. To discover, understand and quan- tify the risks, this paper investigates the prompt- based probing from a causal view, highlights three critical biases which could induce biased results and conclusions, and proposes to con- duct debiasing via causal intervention. This pa- per provides valuable insights for the design of unbiased datasets, better probing frameworks and more reliable evaluations of pretrained lan- guage models. Furthermore, our conclusions also echo that we need to rethink the criteria for identifying better pretrained language models1. 1 Introduction During the past few years, the great success of pretrained language models (PLMs) (Devlin et al., 2019; Liu et al., 2019; Brown et al., 2020; Raffel et al., 2020) raises extensive attention about eval- uating what knowledge do PLMs actually entail. One of the most popular approaches is prompt- based probing (Petroni et al., 2019; Davison et al., 2019; Brown et al., 2020; Schick and Schütze, 2020; Ettinger, 2020; Sun et al., 2021), which assesses whether PLMs are knowledgable for a specific task by querying PLMs with task-specific prompts. For example, to evaluate whether BERT knows the birthplace of Michael Jordan, we could query BERT with “Michael Jordan was born in [MASK]”. Recent studies often construct prompt- based probing datasets, and take PLMs’ perfor- ∗Corresponding Authors 1We openly released the source code and data at https: //github.com/c-box/causalEval. Evaluation Target Language Expression Desirable Effect Linguistic Correlation Test Data Predictor Performance Train Data Hypothesis Architecture Algorithm… Verbalize Predictor PLM Pretrain Data Test Data Performance Prompt Verbalization (b) PLM evaluation via prompt-based probing. Task Distributional Correlation Evaluation Outcome (a) Conventional evaluation in machine learning. Bundled Figure 1: The illustrated procedure for two kinds of evaluation criteria. mance on these datasets as their abilities for the corresponding tasks. Such a probing evaluation has been wildly used in many benchmarks such as SuperGLUE (Wang et al., 2019; Brown et al., 2020), LAMA (Petroni et al., 2019), oLMpics (Tal- mor et al., 2020), LM diagnostics (Ettinger, 2020), CAT (Zhou et al., 2020), X-FACTR (Jiang et al., 2020a), BioLAMA (Sung et al., 2021), etc. Unfortunately, recent studies have found that evaluating PLMs via prompt-based probing …

Content truncated to 500 words