Designing and Interpreting Probes with Control Tasks

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 2733–2743, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 2733 Designing and Interpreting Probes with Control Tasks John Hewitt Stanford University johnhew@stanford.edu Percy Liang Stanford University pliang@cs.stanford.edu Abstract Probes, supervised models trained to pre- dict properties (like parts-of-speech) from representations (like ELMo), have achieved high accuracy on a range of linguistic tasks. But does this mean that the representations encode linguistic structure or just that the probe has learned the linguistic task? In this paper, we propose control tasks, which associate word types with random outputs, to complement linguistic tasks. By construction, these tasks can only be learned by the probe itself. So a good probe, (one that reflects the representation), should be selective, achieving high linguistic task accuracy and low control task accuracy. The selectivity of a probe puts linguistic task accuracy in context with the probe’s capacity to memorize from word types. We construct control tasks for English part-of-speech tagging and dependency edge prediction, and show that popular probes on ELMo representations are not selective. We also find that dropout, commonly used to control probe complexity, is ineffective for improving selectivity of MLPs, but that other forms of regularization are effective. Finally, we find that while probes on the first layer of ELMo yield slightly better part-of-speech tagging accuracy than the second, probes on the second layer are substantially more selective, which raises the question of which layer better represents parts-of-speech. 1 Introduction As large-scale unsupervised representations such as BERT and ELMo improve downstream perfor- mance on a wide range of natural language tasks (Devlin et al., 2019; Peters et al., 2018a; Radford et al., 2019), what these models learn about language remains an open scientific question. An emerging body of work investigates this question through probes, supervised models trained to Sentence 1 The cat ran quickly . Part-of-speech DT NN VBD RB . Control task 10 37 10 15 3 Sentence 2 The dog ran after ! Part-of-speech DT NN VBD IN . Control task 10 15 10 42 42 Figure 1: Our control tasks define random behavior (like a random output, top) for each word type in the vocabulary. Each word token is assigned its type’s output, regardless of context (middle, bottom.) Control tasks have the same input and output space as a linguistic task (e.g., parts-of-speech) but can only be learned if the probe memorizes the mapping. predict a property (like parts-of-speech) from a con- strained view of the representation. Probes trained on various representations have obtained high accuracy on tasks requiring part-of-speech and morphological information (Belinkov et al., 2017), syntactic and semantic information (Peters et al., 2018b; Tenney et al., 2019), among other proper- ties (Conneau et al., 2018), providing evidence that deep representations trained on large datasets are predictive of a broad range of linguistic properties. But when a probe achieves high accuracy on a linguistic task …

Content truncated to 500 words