Does BERT Rediscover a Classical NLP Pipeline?

Proceedings of the 29th International Conference on Computational Linguistics, pages 3143–3153 October 12–17, 2022. 3143 Does BERT Rediscover a Classical NLP Pipeline? Jingcheng Niu Wenjie Lu Gerald Penn University of Toronto Vector Institute {niu,luwenjie,gpenn}@cs.toronto.edu Abstract Does BERT store surface knowledge in its bot- tom layers, syntactic knowledge in its middle layers, and semantic knowledge in its upper lay- ers? In re-examining Jawahar et al. (2019) and Tenney et al.’s (2019a) probes into the structure of BERT, we have found that the pipeline-like separation that they asserted lacks conclusive empirical support. BERT’s structure is, how- ever, linguistically founded, although perhaps in a way that is more nuanced than can be ex- plained by layers alone. We introduce a novel probe, called GridLoc, through which we can also take into account token positions, training rounds, and random seeds. Using GridLoc, we are able to detect other, stronger regularities that suggest that pseudo-cognitive appeals to layer depth may not be the preferable mode of explanation for BERT’s inner workings. 1 Introduction “Surface information at the bottom, syntactic in- formation in the middle, semantic information at the top” (Jawahar et al., 2019). This conclusion is also drawn by Tenney et al. (2019a). While this portrait of multiple layers of linguistic structure has indeed been projected into pipline architectures by some NLP systems before (e.g., Manning et al., 2014), the projection of this understanding onto the internal structure of BERT is now widely ac- cepted both in the BERTology community (Tenney et al., 2019a,b; Hewitt and Liang, 2019; Zhu et al., 2022) and by researchers working on downstream application tasks with BERT (Xiao et al., 2021). At the same time, there is nevertheless scepti- cism about the premises of probing itself. As He- witt and Liang (2019) pondered: “when a probe achieves high accuracy on a linguistic task using a representation, can we conclude that the repre- sentation encodes linguistic structure, or has the probe just learned the task?” Furthermore, Pi- mentel et al. (2020) “cast doubt on whether prob- ing makes sense as a scientific endeavour,” because from an information-theoretic perspective, BERT cannot introduce new linguistic information by pro- cessing the input sequence. In our view, this debate does not need to hin- der the endeavour of uncovering structure within BERT. Regardless of stance, all parties agree with the existence and importance of different levels of information within BERT itself or its embeddings. There is, however, room for improvement in the investigative methods of both Jawahar et al. (2019) and Tenney et al. (2019a) (J&T), which seem to have been limited to observational confirmations of what they sought to find. Our own exploratory analysis has revealed that BERT is indeed linguistically founded, although not in a way that suggests a classical pipeline ar- chitecture, other than what factors through our own functional understanding of NLP’s terminology and subtasks. In addition to examining BERT layers, as J&T did, we have also examined BERT’s structure through the lenses of the choice of random seed, training iterations, …

Content truncated to 500 words