How Pre-trained Language Models Capture Factual Knowledge? A Causal-Inspired Analysis

Findings of the Association for Computational Linguistics: ACL 2022, pages 1720 - 1732 May 22-27, 2022 c⃝2022 Association for Computational Linguistics How Pre-trained Language Models Capture Factual Knowledge? A Causal-Inspired Analysis Shaobo Li1∗, Xiaoguang Li2∗†, Lifeng Shang2, Zhenhua Dong2, Chengjie Sun1†, Bingquan Liu1, Zhenzhou Ji1, Xin Jiang2 and Qun Liu2 1Harbin Institute of Technology 2Huawei Noah’s Ark Lab shli@insun.hit.edu.cn, {sunchengjie, liubq, jizhenzhou}@hit.edu.cn {lixiaoguang11, shang.lifeng, dongzhenhua, Jiang.Xin, qun.liu}@huawei.com Abstract Recently, there has been a trend to investigate the factual knowledge captured by Pre-trained Language Models (PLMs). Many works show the PLMs’ ability to fill in the missing factual words in cloze-style prompts such as “Dante was born in [MASK].” However, it is still a mystery how PLMs generate the results cor- rectly: relying on effective clues or short- cut patterns? We try to answer this ques- tion by a causal-inspired analysis that quan- titatively measures and evaluates the word- level patterns that PLMs depend on to gener- ate the missing words. We check the words that have three typical associations with the missing words: knowledge-dependent, posi- tionally close, and highly co-occurred. Our analysis shows: (1) PLMs generate the miss- ing factual words more by the positionally close and highly co-occurred words than the knowledge-dependent words; (2) the depen- dence on the knowledge-dependent words is more effective than the positionally close and highly co-occurred words. Accordingly, we conclude that the PLMs capture the factual knowledge ineffectively because of depending on the inadequate associations. 1 Introduction d Do Pre-trained Language Models (PLMs) capture factual knowledge? LAMA benchmark (Petroni et al., 2019) answers this question by quantita- tively measuring the factual knowledge captured in PLMs: query PLMs with cloze-style prompts such as “Dante was born in [MASK]?” Filling in the mask with the correct word “Florence” is con- sidered a successful capture of the corresponding factual knowledge. The percentage of correct fill- ings over all the prompts can be used to estimate the amount of factual knowledge captured. PLMs ∗Authors contribute equally. †Corresponding authors: sunchengjie@hit.edu.cn, lixi- aoguang11@huawei.com Knowledge-Dependent: Columbus born between 25 August and 31 October 1451, died 20 May 1506 was an Italian explorer. Positionally Close: Columbus born between 25 August and 31 October 1451, died 20 May 1506 was an Italian explorer. Highly Co-occurred: Columbus born between 25 August and 31 October 1451, died 20 May 1506 was an Italian explorer. Figure 1: The associations we investigated. The un- derlined words are the missing words that need to be generated. The bold words, which hold specific asso- ciations with the missing words, are considered as the word-level patterns that PLMs may use to generate the missing words. show a surprisingly strong ability to capture fac- tual knowledge in such probings (Jiang et al., 2020; Shin et al., 2020; Zhong et al., 2021), which elicits further research on a more in-depth question (Cao et al., 2021; Elazar et al., 2021a): How do PLMs capture the factual knowledge? In this paper, we try to answer this question with a two-fold analysis: Research Question 1 …

Content truncated to 500 words