DirectProbe: Studying Representations without Classifiers

DIRECTPROBE: Studying Representations without Classifiers Yichu Zhou School of Computing University of Utah flyaway@cs.utah.edu Vivek Srikumar School of Computing University of Utah svivek@cs.utah.edu Abstract Understanding how linguistic structure is en- coded in contextualized embedding could help explain their impressive performance across NLP. Existing approaches for probing them usually call for training classifiers and use the accuracy, mutual information, or complex- ity as a proxy for the representation’s good- ness. In this work, we argue that doing so can be unreliable because different representations may need different classifiers. We develop a heuristic, DIRECTPROBE, that directly stud- ies the geometry of a representation by build- ing upon the notion of a version space for a task. Experiments with several linguistic tasks and contextualized embeddings show that, even without training classifiers, DIRECTPROBE can shine light into how an embedding space repre- sents labels, and also anticipate classifier per- formance for the representation. 1 Introduction Distributed representations of words (e.g., Peters et al., 2018; Devlin et al., 2019) have propelled the state-of-the-art across NLP to new heights. Recently, there is much interest in probing these opaque representations to understand the informa- tion they bear (e.g., Kovaleva et al., 2019; Conneau et al., 2018; Jawahar et al., 2019). The most com- monly used strategy calls for training classifiers on them to predict linguistic properties such as syntax, or cognitive skills like numeracy (e.g. Kassner and Schütze, 2020; Perone et al., 2018; Yaghoobzadeh et al., 2019; Krasnowska-Kiera´s and Wróblewska, 2019; Wallace et al., 2019; Pruksachatkun et al., 2020). Using these classifiers, criteria such as accu- racy or model complexity are used to evaluate the representation quality for the task (e.g. Goodwin et al., 2020; Pimentel et al., 2020a; Michael et al., 2020). Such classifier-based probes are undoubtedly useful to estimate a representation’s quality for a task. However, their ability to reveal the informa- tion in a representation is occluded by numerous factors, such as the choice of the optimizer and the initialization used to train the classifiers. For exam- ple, in our experiments using the task of preposition supersense prediction (Schneider et al., 2018), we found that the accuracies across different training runs of the same classifier can vary by as much as ∼ 8%! (Detailed results can be found in Ap- pendix F.) Indeed, the very choice of a classifier influences our estimate of the quality of a representation. For example, one representation may achieve the best classification accuracy with a linear model, whereas another may demand a multi-layer per- ceptron for its non-linear decision boundaries. Of course, enumerating every possible classifier for a task is untenable. A common compromise in- volves using linear classifiers to probe representa- tions (Alain and Bengio, 2017; Kulmizev et al., 2020), but doing so may mischaracterize repre- sentations that need non-linear separators. Some work recognizes this problem (Hewitt and Liang, 2019) and proposes to report probing results for at least logistic regression and a multi-layer percep- tron (Eger et al., 2019), or to compare the learning curves between multiple …

Content truncated to 500 words