Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4166–4176 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 4166 Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT Zhiyong Wu1, Yun Chen2, Ben Kao1, Qun Liu3 1The University of Hong Kong, Hong Kong, China 2Shanghai University of Finance and Economics, Shanghai, China 3Huawei Noah’s Ark Lab, Hong Kong, China {zywu,kao}@cs.hku.hk, yunchen@sufe.edu.cn, qun.liu@huawei.com Abstract By introducing a small set of additional pa- rameters, a probe learns to solve specific lin- guistic tasks (e.g., dependency parsing) in a supervised manner using feature representa- tions (e.g., contextualized embeddings). The effectiveness of such probing tasks is taken as evidence that the pre-trained model en- codes linguistic knowledge. However, this ap- proach of evaluating a language model is un- dermined by the uncertainty of the amount of knowledge that is learned by the probe it- self. Complementary to those works, we pro- pose a parameter-free probing technique for analyzing pre-trained language models (e.g., BERT). Our method does not require direct su- pervision from the probing tasks, nor do we introduce additional parameters to the prob- ing process. Our experiments on BERT show that syntactic trees recovered from BERT us- ing our method are significantly better than linguistically-uninformed baselines. We fur- ther feed the empirically induced dependency structures into a downstream sentiment classi- fication task and find its improvement compat- ible with or even superior to a human-designed dependency schema. 1 Introduction Recent prevalent pre-trained language models such as ELMo (Peters et al., 2018b), BERT (De- vlin et al., 2018), and XLNet (Yang et al., 2019) achieve state-of-the-art performance for a diverse array of downstream NLP tasks. An interesting area of research is to investigate the interpretabil- ity of these pre-trained models (i.e., the linguistic properties they capture). Most recent approaches are built upon the idea of probing classifiers (Shi et al., 2016; Adi et al., 2017; Conneau et al., 2018; Peters et al., 2018a; Hewitt and Manning, 2019; Clark et al., 2019; Tenney et al., 2019b; Jawahar et al., 2019). A probe is a simple neural net- work (with a small additional set of parameters) that uses the feature representations generated by a pre-trained model (e.g., hidden state activations, attention weights) and is trained to perform a su- pervised task (e.g., dependency labeling). The per- formance of a probe is used to measure the qual- ity of the generated representations with the as- sumption that the measured quality is mostly at- tributable to the pre-trained language model. One downside of such approach, as pointed out in (Hewitt and Liang, 2019), is that a probe intro- duces a new set of additional parameters, which makes the results difficult to interpret. Is it the pre- trained model that captures the linguistic informa- tion, or is it the probe that learns the downstream task itself and thus encodes the information in its additional parameter space? In this paper we propose a parameter-free prob- ing technique called Perturbed Masking to analyze and interpret …

Content truncated to 500 words