What if This Modified That? Syntactic Interventions with Counterfactual Embeddings

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 862–875 August 1–6, 2021. ©2021 Association for Computational Linguistics 862 What if This Modified That? Syntactic Interventions via Counterfactual Embeddings Mycal Tucker MIT mycal@mit.edu Peng Qian MIT pqian@mit.edu Roger P. Levy MIT rplevy@mit.edu Abstract Neural language models exhibit impressive performance on a variety of tasks, but their in- ternal reasoning may be difficult to understand. Prior art aims to uncover meaningful proper- ties within model representations via probes, but it is unclear how faithfully such probes portray information that the models actually use. To overcome such limitations, we pro- pose a technique, inspired by causal analy- sis, for generating counterfactual embeddings within models. In experiments testing our technique, we produce evidence that suggests some BERT-based models use a tree-distance- like representation of syntax in downstream prediction tasks. 1 Introduction Large neural models like BERT and GPT-3 have established a new state of the art in a variety of challenging linguistic tasks (Devlin et al., 2019; Brown et al., 2020). These connectionist models, trained on large corpora in a largely unsupervised manner, learn to map words into numerical repre- sentations, or embeddings, that support language- reasoning tasks. Fine-tuning these models on tasks like extractive question answering specializes these generic models into performant, task-specific mod- els (Wolf et al., 2019). In conjunction with the rise of these powerful neural models, researchers have investigated what the models have learned. Probes, tools built to re- veal properties of a trained model, are a favored ap- proach (Hall Maudslay et al., 2020; Conneau et al., 2018). For example, Hewitt and Manning (2019) have uncovered compelling evidence that several models encode syntactic information in their em- beddings. That is, by passing embeddings through a trained probe, one may recover information about a sentence’s syntax. “I saw the boy and the girl [MASK] tall.” M “was” “were” Probe z′ Figure 1: A language model, M, outputs predictions and a probe estimates properties from the model rep- resentation. We use probes to generate counterfactual representations, z′, based on syntactic manipulations, revealing reasoning within the model. Although these results are impressive, they fall short of clearly demonstrating what linguistic infor- mation the language models actually use. Syntactic information is present in sentences; that embed- dings also encode syntax does not imply that a model uses syntactic knowledge. In order to truly query a model’s understanding, one must use causal analysis. Recently, several authors have done so by generating counterfactual data to test models (Kaushik et al., 2020; Goyal et al., 2019; Elazar et al., 2020). They either create new input data or ablate parts of embeddings and study how model outputs change. We extend this prior art via a new technique for generating counter- factual embeddings by using traditional probes to manipulate embeddings according to syntactic prin- ciples, as depicted in Figure 1. Because we conduct experiments with syntactically ambiguous inputs, we are able to measure how models respond to dif- ferent valid parses of the same sentence instead …

Content truncated to 500 words