Naturalistic Causal Probing for Morpho-Syntax

Naturalistic Causal Probing for Morpho-Syntax Afra Amini1,2 Tiago Pimentel3 Clara Meister1 Ryan Cotterell1,2 1ETH Z¨urich, Switzerland 2ETH AI Center, Switzerland 3University of Cambridge, UK afra.amini@inf.ethz.ch tp472@cam.ac.uk clara.meister@inf.ethz.ch ryan.cotterell@inf.ethz.ch Abstract Probing has become a go-to methodology for interpreting and analyzing deep neural mod- els in natural language processing. However, there is still a lack of understanding of the limitations and weaknesses of various types of probes. In this work, we suggest a strat- egy for input-level intervention on naturalistic sentences. Using our approach, we intervene on the morpho-syntactic features of a sen- tence, while keeping the rest of the sentence unchanged. Such an intervention allows us to causally probe pre-trained models. We apply our naturalistic causal probing frame- work to analyze the effects of grammatical gender and number on contextualized rep- resentations extracted from three pre-trained models in Spanish, the multilingual versions of BERT, RoBERTa, and GPT-2. Our exper- iments suggest that naturalistic interventions lead to stable estimates of the causal effects of various linguistic properties. Moreover, our experiments demonstrate the importance of naturalistic causal probing when analyzing pre-trained models. https://github.com/rycolab /naturalistic-causal-probing 1 Introduction Contextualized word representations are a by- product of pre-trained neural language models and have led to improvements in performance on a myriad of downstream natural language process- ing (NLP) tasks (Joshi et al., 2019; Kondratyuk, 2019; Zellers et al., 2019; Brown et al., 2020). Despite this performance improvement, though, it is still not obvious to researchers how these rep- resentations encode linguistic information. One prominent line of work attempts to shed light on this topic through probing (Alain and Bengio, 2017), also referred to as auxiliary prediction (Adi et al., 2017) or diagnostic classification (Hupkes et al., 2018). In machine learning parlance, a probe is a supervised classifier that is trained to predict a property of interest from the target model’s rep- resentations. If the probe manages to predict the property with high accuracy, one may conclude that these representations encode information about the probed property. While widely used, probing is not without its limitations.1 For instance, probing a pre-trained model for grammatical gender can only tell us whether information about gender is present in the representations,2 it cannot, however, tell us how or if the model actually uses information about gender in its predictions (Ravichander et al., 2021; Elazar et al., 2021; Ravfogel et al., 2021; Lasri et al., 2022). Furthermore, supervised prob- ing cannot tell us whether the property under consideration is directly encoded in the represen- tations, or if it can be recovered from the represen- tations alone due to spurious correlations among various linguistic properties. In other words, while we might find correlations between a probed property and representations through supervised probing techniques, we cannot uncover causal relationships between them. In this work, we propose a new strategy for input-level intervention on naturalistic data to ob- tain what we call naturalistic counterfactuals, which we then use to perform causal probing. Through such input-level interventions, we can ascertain whether a particular linguistic property has …

Content truncated to 500 words