Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 1828–1843 August 1–6, 2021. ©2021 Association for Computational Linguistics 1828 Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models Matthew Finlayson∗ Harvard University Cambridge, MA mattbnfin@gmail.com Aaron Mueller∗ Johns Hopkins University Baltimore, MD amueller@jhu.edu Sebastian Gehrmann Google Research New York, NY gehrmann@google.com Stuart Shieber Harvard University Cambridge, MA shieber@seas.harvard.edu Tal Linzen† New York University New York, NY linzen@nyu.edu Yonatan Belinkov‡ Technion – IIT Haifa, Israel belinkov@technion.ac.il Abstract Targeted syntactic evaluations have demon- strated the ability of language models to per- form subject-verb agreement given difficult contexts. To elucidate the mechanisms by which the models accomplish this behavior, this study applies causal mediation analysis to pre-trained neural language models. We inves- tigate the magnitude of models’ preferences for grammatical inflections, as well as whether neurons process subject-verb agreement simi- larly across sentences with different syntactic structures. We uncover similarities and differ- ences across architectures and model sizes— notably, that larger models do not necessar- ily learn stronger preferences. We also ob- serve two distinct mechanisms for producing subject-verb agreement depending on the syn- tactic structure of the input sentence. Finally, we find that language models rely on similar sets of neurons when given sentences with sim- ilar syntactic structure. 1 Introduction Targeted syntactic evaluations have shown that neu- ral language models (LMs) are able to predict the correct token from a set of grammatically mini- mally different continuations with high accuracy, even in difficult contexts (Linzen et al., 2016; Gu- lordava et al., 2018), for constructions such as subject-verb agreement (van Schijndel et al., 2019), filler-gap dependencies (Wilcox et al., 2018), and reflexive anaphora (Marvin and Linzen, 2018). As an illustration of the targeted syntactic eval- uation paradigm, consider the following example, which demonstrates subject-verb agreement across an agreement attractor. Here, a model using a linear ∗Equal contribution. †Work done while visiting Google Research. ‡ Supported by the Viterbi Fellowship in the Center for Computer Engineering at the Technion. analysis (i.e., inflecting based on the most recent noun) would choose the ungrammatical inflection, while a model using a hierarchical analysis would choose the grammatical inflection: (1) The key to the cabinets is/*are next to the coins. While we have a reasonable understanding of the generally correct behavior of LMs in such con- texts, the mechanisms that underlie models’ sensi- tivity to syntactic agreement are still not well under- stood. Recent work has performed causal analyses of syntactic agreement units in LSTM (Hochre- iter and Schmidhuber, 1997)-based LMs (Lakretz et al., 2019; Lu et al., 2020) or causal analyses of LSTM hidden representations’ impact on syntactic agreement (Giulianelli et al., 2018), but the agree- ment mechanisms of Transformer-based LMs have not been as extensively investigated. Transformer- based LMs’ syntactic generalization abilities are superior to those of LSTMs (Hu et al., 2020), which makes Transformer-based models enticing candi- dates for further analysis. We apply the behavioral-structural method of causal …

Content truncated to 500 words