Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Prediction

Proceedings of the 25th Conference on Computational Natural Language Learning (CoNLL), pages 194–209 November 10–11, 2021. ©2021 Association for Computational Linguistics 194 Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Prediction Shauli Ravfogel∗1,2 Grusha Prasad∗3 Tal Linzen4 Yoav Goldberg1,2 1Computer Science Department, Bar Ilan University 2Allen Institute for Artificial Intelligence 3Cognitive Science Department, Johns Hopkins University 4Department of Linguistics and Center for Data Science, New York University linzen@nyu.edu, grusha.prasad@jhu.edu {shauli.ravfogel, yoav.goldberg}@gmail.com Abstract When language models process syntactically complex sentences, do they use their represen- tations of syntax in a manner that is consistent with the grammar of the language? We pro- pose AlterRep, an intervention-based method to address this question. For any linguistic feature of a given sentence, AlterRep gener- ates counterfactual representations by altering how the feature is encoded, while leaving in- tact all other aspects of the original represen- tation. By measuring the change in a model’s word prediction behavior when these counter- factual representations are substituted for the original ones, we can draw conclusions about the causal effect of the linguistic feature in question on the model’s behavior. We apply this method to study how BERT models of different sizes process relative clauses (RCs). We find that BERT variants use RC boundary information during word prediction in a man- ner that is consistent with the rules of English grammar; this RC boundary information gen- eralizes to a considerable extent across differ- ent RC types, suggesting that BERT represents RCs as an abstract linguistic category. 1 Introduction The success of neural language models, both in NLP tasks and as cognitive models, has fueled tar- geted evaluation of these models’ word prediction accuracy on a range of syntactically complex con- structions (Linzen et al., 2016; Gauthier et al., 2020; Warstadt et al., 2020; Mueller et al., 2020; Marvin and Linzen, 2018). What are the internal repre- sentations that support such sophisticated syntactic behavior? In this paper, we tackle this question using an intervention-based approach (Woodward, 2005). Our method, AlterRep, is designed to study whether a model uses a particular linguistic feature in a manner which is consistent with the grammar of the language. The method involves two steps: ∗Equal contribution. first, it generates counterfactual1 contextual word representations by altering the neural network’s rep- resentation of the linguistic feature under consid- eration; and second, it characterizes the change in the model’s word prediction behaviour that results from replacing the original word representations with their counterfactual variants. If the resulting change in word prediction aligns with predictions from linguistic theory, we can infer that the model uses the feature under consideration in a manner consistent with the grammar of the language. We demonstrate the utility of AlterRep using rel- ative clauses (RCs). According to the grammar of English, to correctly determine whether the masked verb in (1) should be singular or plural, a model must recognize that the masked verb is outside the RC the officers love, and should therefore agree with the subject of the main …

Content truncated to 500 words