BERT Busters: Outlier Dimensions that Disrupt Transformers

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3392–3405 August 1–6, 2021. ©2021 Association for Computational Linguistics 3392 BERT Busters: Outlier Dimensions that Disrupt Transformers Olga Kovaleva∗1, Saurabh Kulshreshtha∗1, Anna Rogers2 and Anna Rumshisky1 1Department of Computer Science, University of Massachusetts Lowell 2Center for Social Data Science, University of Copenhagen 1{okovalev,skul,arum}@{cs.uml.edu} 2arogers@sodas.ku.dk Abstract Multiple studies have shown that Transform- ers are remarkably robust to pruning. Con- trary to this received wisdom, we demon- strate that pre-trained Transformer encoders are surprisingly fragile to the removal of a very small number of features in the layer out- puts (<0.0001% of model weights). In case of BERT and other pre-trained encoder Trans- formers, the affected component is the scal- ing factors and biases in the LayerNorm. The outliers are high-magnitude normalization pa- rameters that emerge early in pre-training and show up consistently in the same dimensional position throughout the model. We show that disabling them significantly degrades both the MLM loss and the downstream task perfor- mance. This effect is observed across sev- eral BERT-family models and other popular pre-trained Transformer architectures, includ- ing BART, XLNet and ELECTRA; we also show a similar effect in GPT-2. 1 Introduction Pre-trained Transformer-based models (Vaswani et al., 2017) have become widely popular in a variety of NLP applications. Multiple studies of BERT-family models (Devlin et al., 2019) showed that Transformers are remarkably robust to prun- ing (Gordon et al., 2020; Prasanna et al., 2020; Chen et al., 2020; Michel et al., 2019). This work presents a different and unexpected result: it is pos- sible to dramatically disrupt the performance of BERT and other Transformer-based architectures by modifying very few weights (less than 0.0001% for BERT). In particular, we show that there is a very small number of outlier dimensions that regularly appear in the same position in the pre-trained encoder lay- ers of a given Transformer model. We demonstrate ∗ Authors contributed equally to this work. that this effect holds for different Transformer- family architectures, including multiple variants of BERT, as well as ELECTRA (Clark et al., 2020), BART (Lewis et al., 2020), and XLNet (Yang et al., 2019). A similar phenomenon is also present in the decoder layers of GPT-2 (Radford et al.). When these dimensions are disabled throughout the model in the concluding transformation of each layer, they can drastically reduce the overall model performance. With the exception of GPT-2, the last transformation in each layer of these models is normalization (LayerNorm), which is what we mainly focus on in this study. The contributions of this work are as follows: • We identify certain outlier dimensions in Transformer layer outputs and show that they play a crucial role in both language modeling and downstream task performance. Disabling the weights for these output dimensions drasti- cally degrades performance (up to 44 points). • We show that this effect holds for the encoder layers of six different models of the BERT family, as well as other popular pre-trained Transformer-based models including ELEC- TRA, BART, …

Content truncated to 500 words