Information-Theoretic Probing for Linguistic Structure

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4609–4622 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 4609 Information-Theoretic Probing for Linguistic Structure Tiago PimentelD Josef ValvodaD Rowan Hall MaudslayD Ran ZmigrodD Adina Williams@ Ryan CotterellD,Q DUniversity of Cambridge @Facebook AI Research QETH Zürich tp472@cam.ac.uk, jv406@cam.ac.uk, rh635@cam.ac.uk, rz279@cam.ac.uk, adinawilliams@fb.com, rdc42@cl.cam.ac.uk Abstract The success of neural networks on a diverse set of NLP tasks has led researchers to ques- tion how much these networks actually “know” about natural language. ural way of assessing this.Probes are a nat- When probing, a researcher chooses a linguistic task and trains a supervised model to predict annota- tions in that linguistic task from the network’s learned representations. If the probe does well, the researcher may conclude that the representations encode knowledge related to the task. A commonly held belief is that us- ing simpler models as probes is better; the logic is that simpler models will identify lin- guistic structure, but not learn the task it- self. We propose an information-theoretic op- erationalization of probing as estimating mu- tual information that contradicts this received wisdom: one should always select the high- est performing probe one can, even if it is more complex, since it will result in a tighter estimate, and thus reveal more of the lin- guistic information inherent in the represen- tation. The experimental portion of our pa- per focuses on empirically estimating the mu- tual information between a linguistic property and BERT, comparing these estimates to sev- eral baselines. We evaluate on a set of ten typologically diverse languages often under- represented in NLP research—plus English— totalling eleven languages. Our implementa- tion is available in https://github.com/ rycolab/info-theoretic-probing. 1 Introduction Neural networks are the backbone of modern state- of-the-art natural language processing (NLP) sys- tems. One inherent by-product of training a neural network is the production of real-valued represen- tations. Many speculate that these representations encode a continuous analogue of discrete linguis- tic properties, e.g., part-of-speech tags, due to the networks’ impressive performance on many NLP tasks (Belinkov et al., 2017). As a result of this speculation, one common thread of research fo- cuses on the construction of probes, i.e., super- vised models that are trained to extract the linguis- tic properties directly (Belinkov et al., 2017; Con- neau et al., 2018; Peters et al., 2018b; Zhang and Bowman, 2018; Naik et al., 2018; Tenney et al., 2019). A syntactic probe, then, is a model for ex- tracting syntactic properties, such as part of speech, from the representations (Hewitt and Liang, 2019). In this work, we question what the goal of prob- ing for linguistic properties ought to be. Infor- mally, probing is often described as an attempt to discern how much information representations encode about a specific linguistic property. We make this statement more formal: We assert that the natural operationalization of probing is estimat- ing the mutual information (Cover and Thomas, 2012) between a representation-valued random variable and a linguistic property–valued random variable. This …

Content truncated to 500 words