Conditional probing: measuring usable information beyond a baseline

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1626–1639 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 1626 Conditional probing: measuring usable information beyond a baseline John Hewitt Kawin Ethayarajh Percy Liang Department of Computer ScienceChristopher D. Manning Stanford University {johnhew,kawin,pliang,manning}@cs.stanford.edu Abstract Probing experiments investigate the ex- tent to which neural representations make properties—like part-of-speech—predictable. One suggests that a representation encodes a property if probing that representation produces higher accuracy than probing a baseline representation like non-contextual word embeddings. Instead of using baselines as a point of comparison, we’re interested in measuring information that is contained in the representation but not in the baseline. For example, current methods can detect when a representation is more useful than the word identity (a baseline) for predicting part-of- speech; however, they cannot detect when the representation is predictive of just the aspects of part-of-speech not explainable by the word identity. In this work, we extend a theory of usable information called V-information and propose conditional probing, which explicitly conditions on the information in the baseline. In a case study, we find that after conditioning on non-contextual word embeddings, properties like part-of-speech are accessible at deeper layers of a network than previously thought. 1 Introduction Neural language models have become the foun- dation for modern NLP systems (Devlin et al., 2019; Radford et al., 2018), but what they under- stand about language, and how they represent that knowledge, is still poorly understood (Belinkov and Glass, 2019; Rogers et al., 2020). The prob- ing methodology grapples with these questions by relating neural representations to well-understood properties. Probing analyzes a representation by us- ing it as input into a supervised classifier, which is trained to predict a property, such as part-of-speech (Shi et al., 2016; Ettinger et al., 2016; Alain and Bengio, 2016; Adi et al., 2017; Belinkov, 2021). One suggests that a representation encodes a property of interest if probing that representation produces higher accuracy than probing a baseline representation like non-contextual word embed- dings. However, consider a representation that encodes only the part-of-speech tags that aren’t determined by the word identity. Probing would report that this representation encodes less about part-of-speech than the non-contextual word base- line, since ambiguity is relatively rare. Yet, this representation clearly encodes interesting aspects of part-of-speech. How can we capture this? In this work, we present a simple probing method to explicitly condition on a baseline.1 For a representation and a baseline, our method trains two probes: (1) on just the baseline, and (2) on the concatenation of the baseline and the represen- tation. The performance of probe (1) is then sub- tracted from that of probe (2). We call this process conditional probing. Intuitively, the representation is not penalized for lacking aspects of the property accessible in the baseline. We then theoretically ground our probing methodology in V-information, a theory of usable information introduced by Xu et al. (2020) that we additionally extend to multiple predictive vari- ables. We use V-information instead of …

Content truncated to 500 words