On the data requirements of probing

Findings of the Association for Computational Linguistics: ACL 2022, pages 4132 - 4147 May 22-27, 2022 c⃝2022 Association for Computational Linguistics On the data requirements of probing Zining Zhu1,2, Jixuan Wang1,2, Bai Li1,2, Frank Rudzicz1,2,3 1 University of Toronto 2 Vector Institute for Artificial Intelligence 3 Unity Health Toronto {zining,jixuan,bai,frank}@cs.toronto.edu Abstract As large and powerful neural language models are developed, researchers have been increas- ingly interested in developing diagnostic tools to probe them. There are many papers with con- clusions of the form “observation X is found in model Y ”, using their own datasets with vary- ing sizes. Larger probing datasets bring more reliability, but are also expensive to collect. There is yet to be a quantitative method for es- timating reasonable probing dataset sizes. We tackle this omission in the context of compar- ing two probing configurations: after we have collected a small dataset from a pilot study, how many additional data samples are sufficient to distinguish two different configurations? We present a novel method to estimate the required number of data samples in such experiments and, across several case studies, we verify that our estimations have sufficient statistical power. Our framework helps to systematically con- struct probing datasets to diagnose neural NLP models. 1 Introduction While modern deep neural language models achieve impressive performance on various bench- marking datasets, the question of how this is achieved is gaining increased attention. This line of inquiry includes a new avenue of research: probing. There are many methods to probe a neural net- work. Among these, diagnostic classification is by far the most common. To probe a neural net- work in a classification configuration, we specify a classification task that examines an ability (e.g., detecting verb tense). We encode the texts with a deep neural network and apply a post-hoc classi- fier to the encoded representations. If the classifier can easily predict an attribute, we consider this deep neural network capable of encoding the speci- fied ability. Researchers have expanded the targets of probing classifications to a wide range of abili- ties including syntax and semantics (Jawahar et al., Figure 1: Sizes of the datasets in some common probing suites. Depending on the tasks, they vary from hundreds to larger than 105. 2019; Tenney et al., 2019a; Kulmizev et al., 2020; Vuli´c et al., 2020), discourse (Koto et al., 2021; Zhu et al., 2020; Chen et al., 2019) and commonsense reasoning (Petroni et al., 2019; Lin et al., 2020). These probing papers are associated with datasets of varying sizes, as shown in Figure 1. What is a suitable size for probing datasets? Larger training datasets lead to tighter generaliza- tion bounds. With other conditions fixed, larger testing datasets allow for higher statistical power in comparing probing models and classifiers. That said, it is neither realistic nor desirable to increase the probing dataset sizes arbitrarily. It is therefore essential to find a balanced size for probing classi- fications. We propose a framework to formally estimate the data requirements of probing configurations …

Content truncated to 500 words