Classifier Probes May Just Learn from Linear Context Features

Proceedings of the 28th International Conference on Computational Linguistics, pages 5136–5146 Barcelona, Spain (Online), December 8-13, 2020 5136 Classifier Probes May Just Learn from Linear Context Features Jenny Kunz and Marco Kuhlmann Dept. of Computer and Information Science Linköping University jenny.kunz@liu.se and marco.kuhlmann@liu.se Abstract Classifiers trained on auxiliary probing tasks are a popular tool to analyze the representations learned by neural sentence encoders such as BERT and ELMo. While many authors are aware of the difficulty to distinguish between “extracting the linguistic structure encoded in the representations” and “learning the probing task,” the validity of probing methods calls for further research. Using a neighboring word identity prediction task, we show that the token embeddings learned by neural sentence encoders contain a significant amount of information about the exact linear context of the token, and hypothesize that, with such information, learning standard probing tasks may be feasible even without additional linguistic structure. We develop this hypothesis into a framework in which analysis efforts can be scrutinized and argue that, with current models and baselines, conclusions that representations contain linguistic structure are not well-founded. Current probing methodology, such as restricting the classifier’s expressiveness or using strong baselines, can help to better estimate the complexity of learning, but not build a foundation for speculations about the nature of the linguistic structure encoded in the learned representations. 1 Introduction The impressive performance of neural language models such as ELMo (Peters et al., 2018) and BERT (Devlin et al., 2019) has led many authors to believe that these models must have learned linguistic structure, and a whole new research branch concerned with the analysis and interpretation of neural models has emerged, with dedicated areas at many conferences and the BlackboxNLP workshop being established as a large and well-known venue in a short time. An appealing and widespread analysis technique, perhaps due to its simplicity and generalizability, is to use a model’s word representations as input to a simple classifier, and train this classifier on an auxiliary task that can be expected to benefit from linguistic knowledge. The argument behind this technique is that, if a simple classifier is enough to learn the task based on word-level representations alone, then it is probable that these representations indeed encode the hypothesized linguistic structure (Hupkes et al., 2018). Many of the papers applying classifier probes suggest that models such as BERT and ELMo encode structure that is similar to well-known syntactic annotations, e.g. dependency trees (Hewitt and Manning, 2019), or even to a traditional NLP pipeline (Tenney et al., 2019a). A standard assumption in probing is that, if we feed only the representation of one word at a time to the classifier, then no complex inference based on other words in the sentence can happen, and the probe will not be able to learn the classification task well—unless the necessary linguistic information already is encoded in the representation (Lin et al., 2019). Therefore, the argument goes, high performance on a probing task can be taken as evidence that …

Content truncated to 500 words