A Bayesian Framework for Information-Theoretic Probing

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2869–2887 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 2869 A Bayesian Framework for Information-Theoretic Probing Tiago Pimentel University of Cambridge tp472@cam.ac.uk Ryan Cotterell ETH Zürich ryan.cotterell@inf.ethz.ch Abstract Pimentel et al. (2020b) recently analysed prob- ing from an information-theoretic perspective. They argue that probing should be seen as ap- proximating a mutual information. This led to the rather unintuitive conclusion that rep- resentations encode exactly the same informa- tion about a target task as the original sen- tences. The mutual information, however, as- sumes the true probability distribution of a pair of random variables is known, leading to unintuitive results in settings where it is not. This paper proposes a new framework to mea- sure what we term Bayesian mutual infor- mation, which analyses information from the perspective of Bayesian agents—allowing for more intuitive findings in scenarios with finite data. For instance, under Bayesian MI we have that data can add information, process- ing can help, and information can hurt, which makes it more intuitive for machine learning applications. Finally, we apply our framework to probing where we believe Bayesian mutual information naturally operationalises ease of extraction by explicitly limiting the available background knowledge to solve a task. 1 Introduction Pimentel et al. (2020b) recently undertook an information-theoretic analysis of probing. They ar- gue that probing may be viewed as approximating the mutual information between a linguistic prop- erty (e.g., part-of-speech tags) and a contextual representation (e.g., BERT). Counter-intuitively, however, due to the data-processing inequality, contextual representations contain exactly the same information about any task as the original sentence, under mild conditions. When viewed under this lens, the goal of probing is not inherently clear. One limitation of Pimentel et al.’s analysis is that it focuses on the mutual information (MI)—to be of practical application, their argument re- quires that a probe matches the true distribution 100 101 102 103 104 105 # Data Examples 0 2 4 6 8 Entropy (bits) −Iθ(R → T) Iθ(R → T) I(R; T) Iθ(Dn → Θ) Hθ(T) Hθ(T | R) H(T) H(T | R) Figure 1: A smoothed example of Bayesian MI on de- pendency arc labelling in English. according to which the data were generated in the limit of finite training data. In contrast, our paper formulates an information-theoretic framework that is compatible both with model misspecification and the finite data assumption. In his seminal work, Shannon (1948) occupied himself with the limit of communication. Indeed, mutual information can be described as the theo- retical limit (or upper-bound) of how much infor- mation can be extracted from one random variable about another. However, this limit is only achiev- able when one has full knowledge of these random variables, including the true probability distribution according to which they are distributed. In practice, we will not have access to such information and it may be difficult to approximate. It follows that any system with imperfect knowledge of the ran- dom variable’s …

Content truncated to 500 words