When Does Syntax Mediate Neural Language Model Performance? Evidence from Dropout Probes

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5393 - 5408 July 10-15, 2022 ©2022 Association for Computational Linguistics When Does Syntax Mediate Neural Language Model Performance? Evidence from Dropout Probes Mycal Tucker mycal@mit.edu Tiwalayo Eisape eisape@mit.edu Peng Qian pqian@mit.edu Roger Levy rplevy@mit.edu Julie Shah julie_a_shah@csail.mit.edu Abstract Recent causal probing literature reveals when language models and syntactic probes use simi- lar representations. Such techniques may yield “false negative” causality results: models may use representations of syntax, but probes may have learned to use redundant encodings of the same syntactic information. We demonstrate that models do encode syntactic information redundantly and introduce a new probe design that guides probes to consider all syntactic in- formation present in embeddings. Using these probes, we find evidence for the use of syntax in models where prior methods did not, allow- ing us to boost model performance by injecting syntactic information into representations. 1 Introduction Recent large neural models like BERT and GPT- 3 exhibit impressive performance on a large va- riety of linguistic tasks, from sentiment analysis to question-answering (Devlin et al., 2019; Brown et al., 2020). Given the models’ impressive perfor- mance, but also their complexity, researchers have developed tools to understand what patterns mod- els have learned. In probing literature, researchers develop “probes:” models designed to extract infor- mation from the representations of trained mod- els (Linzen et al., 2016; Conneau et al., 2018; Hall Maudslay et al., 2020). For example, He- witt and Manning (2019) demonstrated that one can train accurate linear classifiers to predict syn- tactic structure from BERT or ELMO embeddings. These probes reveal what information is present in model embeddings but not how or if models use that information (Belinkov, 2021). To address this gap, new research in causal anal- ysis seeks to understand how aspects of models’ representations affect their behavior (Elazar et al., 2020; Ravfogel et al., 2020; Giulianelli et al., 2018; Tucker et al., 2021; Feder et al., 2021). Typically, these techniques create counterfactual representa- tions that differ from the original according to some Figure 1: In a 2D embedding space, a model might re- dundantly encode syntactic representations of a sentence like “the girl saw the boy with the telescope.” Redun- dant encodings could cause misalignment between the model’s decision boundary (blue) and a probe’s (red). We introduce dropout probes (green) to use all informa- tive dimensions. property (e.g., syntactic interpretation of the sen- tence). Researchers then compare outputs when using original and counterfactual embeddings to assess whether a property encoded in the represen- tation is causally related to model behavior. Unfortunately, negative results — wherein re- searchers report that models do not appear to use a property causally — are difficult to interpret. Such failures can be attributed to a model truly not us- ing the property (true negatives), or to a failure of the technique (false negatives). For example, as de- picted in Figure 1, if a language model encodes …

Content truncated to 500 words