Probing as Quantifying Inductive Bias

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 1839 - 1851 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Probing as Quantifying Inductive Bias Alexander Immer∗, Lucas Torroba Hennigen∗, Vincent Fortuin Ryan Cotterell ETH Zürich MIT alexander.immer@inf.ethz.ch lucastor@mit.edu fortuin@inf.ethz.ch ryan.cotterell@inf.ethz.ch Abstract Pre-trained contextual representations have led to dramatic performance improvements on a range of downstream tasks. Such performance improvements have motivated researchers to quantify and understand the linguistic informa- tion encoded in these representations. In gen- eral, researchers quantify the amount of linguis- tic information through probing, an endeavor which consists of training a supervised model to predict a linguistic property directly from the contextual representations. Unfortunately, this definition of probing has been subject to ex- tensive criticism in the literature, and has been observed to lead to paradoxical and counter- intuitive results. In the theoretical portion of this paper, we take the position that the goal of probing ought to be measuring the amount of inductive bias that the representations en- code on a specific task. We further describe a Bayesian framework that operationalizes this goal and allows us to quantify the representa- tions’ inductive bias. In the empirical portion of the paper, we apply our framework to a vari- ety of NLP tasks. Our results suggest that our proposed framework alleviates many previous problems found in probing. Moreover, we are able to offer concrete evidence that—for some tasks—fastText can offer a better inductive bias than BERT.1 1 Introduction Improved pre-trained representations have led to new performance heights on NLP applications. This has prompted researchers to analyze these representations in an attempt to determine which linguistic properties they encode. Probing is the primary method to perform such a quantification; typically, probing consists of training a supervised model, called a probe, to predict a linguistic prop- erty directly from the representations. It has been *Equal contribution. 1Our code is available at https://github.com/ rycolab/evidence-probing. argued that the existence of a high-performing probe suggests that the representation encodes the property of interest (Alain and Bengio, 2017; Be- linkov and Glass, 2019). However, despite the apparent simplicity of probing and its wide-spread use, the community has yet to find consensus on several important problems about the endeavor. We enumerate several problems with the supervised probing framework in the following paragraphs. Problem I (Representation Selection). Counter- intuitively, probing may fail to capture observed differences between representations. For instance, in some supervised probing studies, researchers have shown that random representations are equally good or better than trained ones (Zhang and Bow- man, 2018; Pimentel et al., 2020a). This is certainly a nonsensical result; random representations, by construction, do not encode any linguistic property. Problem II (Probe Selection). There is an ongoing debate on the choice of probes: initially, linear probes were proposed to test the linear sepa- rability of learned representations (Montavon et al., 2011; Alain and Bengio, 2017; Liu et al., 2019a). However, more recently, neural networks have been applied with the explicit goal of …

Content truncated to 500 words