From BERT‘s Point of View: Revealing the Prevailing Contextual Differences

Findings of the Association for Computational Linguistics: ACL 2022, pages 1120 - 1138 May 22-27, 2022 c⃝2022 Association for Computational Linguistics From BERT’s Point of View: Revealing the Prevailing Contextual Differences Carolin M. Schuster Technical University of Munich carolin.schuster@tum.de Simon Hegelich Technical University of Munich Abstract Though successfully applied in research and industry large pretrained language models of the BERT family are not yet fully understood. While much research in the field of BERTol- ogy has tested whether specific knowledge can be extracted from layer activations, we invert the popular probing design to analyze the pre- vailing differences and clusters in BERT’s high dimensional space. By extracting coarse fea- tures from masked token representations and predicting them by probing models with ac- cess to only partial information we can appre- hend the variation from ‘BERT’s point of view’. By applying our new methodology to different datasets we show how much the differences can be described by syntax but further how they are to a great extent shaped by the most simple positional information. 1 Introduction By taking on the perspective of BERT and present- ing the methodology to explore this point of view we contribute a new approach to BERTology re- search, a field that has emerged for a number of good reasons. Ever since the original BERT paper (Devlin et al., 2019) combined masked language model- ing (MLM) with massive pretraining and the trans- former architecture (Vaswani et al., 2017), models of the BERT family have achieved a variety of new Natural Language Processing (NLP) benchmarks. While their success is driven by the contextualiza- tion of words it is clear that these models do not yet have a real understanding of language (Bender and Koller, 2020). Still, researchers are struggling to find out what it is exactly that they learn and how they perform so well. Challenges are the high number of parameters over which the model knowl- edge is distributed, and the innumberable different patterns the models can potentially gather from text. BERTology takes on this quest of understanding the inner workings of these large pretrained mod- els to drive further improvements and identify the next steps towards general AI. Though the train- ing of ever greater models with ever more data has been criticized because of the societal costs and risks these models bring with, including bias and discrimination (Bender et al., 2021). Never- theless, because of their high performance they are already employed in research but also industry- applications, which exacerbates the need for their explainability. The black box of Bidirectional Encoder Repre- sentations from Transformers (BERT) and its rel- atives commonly consists of 6-24 identical trans- former encoder layers. Each layer comprises a multi-head-attention block followed by a fully con- nected block, with both being bypassed by resid- ual connections (Vaswani et al., 2017). This stack of layers is primarily pretrained with MLM, the task of predicting a randomly masked (or replaced) word in an input text, and can afterwards be fine- tuned to specific …

Content truncated to 500 words