An information theoretic view on selecting linguistic probes

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 9251–9262, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 9251 An information theoretic view on selecting linguistic probes Zining Zhu1,2, Frank Rudzicz3,4,1,2 1 University of Toronto, 2 Vector Institute, 3 Surgical Safety Technologies 4 Li Ka Shing Knowledge Institute, St Michael’s Hospital {zining,frank}@cs.toronto.edu Abstract There is increasing interest in assessing the lin- guistic knowledge encoded in neural represen- tations. A popular approach is to attach a diag- nostic classifier – or “probe” – to perform su- pervised classification from internal represen- tations. However, how to select a good probe is in debate. Hewitt and Liang (2019) showed that a high performance on diagnostic classifi- cation itself is insufficient, because it can be attributed to either “the representation being rich in knowledge”, or “the probe learning the task”, which Pimentel et al. (2020) challenged. We show this dichotomy is valid information- theoretically. In addition, we find that the methods to construct and select good probes proposed by the two papers, control task (He- witt and Liang, 2019) and control function (Pi- mentel et al., 2020), are equivalent – the errors of their approaches are identical (modulo irrel- evant terms). Empirically, these two selection criteria lead to results that highly agree with each other. 1 Introduction Recently, neural networks have shown substantial progress in NLP tasks (Devlin et al., 2019; Rad- ford et al., 2019). To understand and explain their behavior, a natural question emerge: how much linguistic knowledge is encoded in these neural network systems? An efficient approach to reveal information en- coded in internal representations uses diagnostic classifiers (Alain and Bengio, 2017). Referred to as “probes”, diagnostic classifiers are trained on pre-computed intermediate representations of neu- ral NLP systems. The performance on tasks they are trained to predict are used to evaluate the rich- ness of the linguistic representation in encoding the probed tasks. Such tasks include probing syn- tax (Hewitt and Manning, 2019; Lin et al., 2019; Tenney et al., 2019a), semantics (Yaghoobzadeh et al., 2019), discourse features (Chen et al., 2019; Liu et al., 2019; Tenney et al., 2019b), and com- monsense knowledge (Petroni et al., 2019; Poerner et al., 2019). However, appropriate criteria for selecting a good probe is under debate. The traditional view that high-accuracy probes are better is challenged by Hewitt and Liang (2019), who proposed that the high accuracy could be attributed to either (1) that the representation contains rich linguistic knowl- edge, or (2) that the probe learns the task. To cir- cumvent this ambiguity, they proposed to use the improvement of probing task performance against a control task (predicting random labels from the same representations), i.e., the “selectivity” crite- rion. Recently, Pimentel et al. (2020), challenged this dichotomy from an information theoretic view- point. They proposed to use an “information gain” criterion, which empirically is the reduction in cross entropy from a “control function task” prob- ing from randomized representation. In this paper, we show the “non-exclusive-or” dichotomy raised …

Content truncated to 500 words