Pareto Probing: Trading Off Accuracy for Complexity

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 3138–3153, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 3138 Pareto Probing: Trading Off Accuracy for Complexity Tiago Pimentel∗, Naomi Saphra∗, Adina Williams Ryan Cotterell , University of Cambridge University of Edinburgh Facebook AI Research ETH Zürich tp472@cam.ac.uk, n.saphra@ed.ac.uk, adinawilliams@fb.com, ryan.cotterell@inf.ethz.ch Abstract The question of how to probe contextual word representations for linguistic structure in a way that is both principled and useful has seen sig- nificant attention recently in the NLP literature. In our contribution to this discussion, we argue for a probe metric that reflects the fundamental trade-off between probe complexity and per- formance: the Pareto hypervolume. To mea- sure complexity, we present a number of para- metric and non-parametric metrics. Our exper- iments using Pareto hypervolume as an evalua- tion metric show that probes often do not con- form to our expectations—e.g., why should the non-contextual fastText representations encode more morpho-syntactic information than the contextual BERT representations? These results suggest that common, simplistic probing tasks, such as part-of-speech labeling and dependency arc labeling, are inadequate to evaluate the linguistic structure encoded in contextual word representations. This leads us to propose full dependency parsing as a probing task. In support of our suggestion that harder probing tasks are necessary, our experiments with dependency parsing reveal a wide gap in syntactic knowledge between contextual and non-contextual representations. Our code can be found at https://github. com/rycolab/pareto-probing. 1 Introduction Neural networks are a pillar of modern NLP sys- tems. However, their inner workings are poorly understood; indeed, for this reason, they are of- ten referred to as black-box systems (Psichogios and Ungar, 1992; Orphanos et al., 1999; Cauer et al., 2000). This lack of understanding, coupled with the rising adoption of neural NLP systems in both industry and academia, has fomented a rapidly growing literature devoted to “cracking open the black box,” as it were (Alishahi et al., 2019; Linzen ∗Equal contribution 0.0 0.2 0.4 0.6 0.8 1.0 Label-shuffled Complexity 0.0 0.2 0.4 0.6 0.8 1.0 UAS Accurate Simple Selective Figure 1: Probe results on dependency parsing in En- glish. The x-axis corresponds to complexity and mea- sures a probe’s ability to memorize the training data. The y-axis measures the probes performance on the task. Probing the representations: ALBERT, BERT, RoBERTa, fastText, one-hot, and random. et al., 2019). One popular method for studying the linguistic content of neural networks is probing, which we define in this work as training a super- vised classifier (known as a probe) on top of pre- trained models’ frozen representations (Alain and Bengio, 2017). By analyzing the classifier’s perfor- mance, one can assess how much ‘knowledge’ the representations contain about language. Much work in probing advocates for the need for simple probes (Hewitt and Manning, 2019; Hall Maudslay et al., 2020). Indeed, on this point, Alain and Bengio (2017) write: “The task of a deep neural network clas- sifier is to come up with a representation for the final layer that can be …

Content truncated to 500 words