Language model acceptability judgements are not always robust to context

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 6043–6063 July 9-14, 2023 ©2023 Association for Computational Linguistics Language model acceptability judgements are not always robust to context Koustuv Sinha ∗,∞ Jon Gauthier ∗,1 Aaron Mueller †,3 Kanishka Misra †,2 Keren Fuentes ∞ Roger Levy 1 Adina Williams ∞ ∞Meta AI; 1MIT 2Purdue University 3Johns Hopkins ∗, † Equal contributions koustuvs@meta.com, jon@gauthiers.net Abstract Targeted syntactic evaluations of language mod- els ask whether models show stable prefer- ences for syntactically acceptable content over minimal-pair unacceptable inputs. Our best syntactic evaluation datasets, however, provide substantially less linguistic context than mod- els receive during pretraining. This mismatch raises an important question: how robust are models’ syntactic judgements across different contexts? In this paper, we vary the input con- texts based on: length, the types of syntac- tic phenomena it contains, and whether or not there are grammatical violations. We find that model judgements are generally robust when placed in randomly sampled linguistic contexts, but are unstable when contexts match the test stimuli in syntactic structure. Among all tested models (GPT-2 and five variants of OPT), we find that model performance is affected when we provided contexts with matching syntactic structure: performance significantly improves when contexts are acceptable, and it signifi- cantly declines when they are unacceptable. This effect is amplified by the length of the con- text, except for unrelated inputs. We show that these changes in model performance are not ex- plainable by acceptability-preserving syntactic perturbations. This sensitivity to highly spe- cific syntactic features of the context can only be explained by the models’ implicit in-context learning abilities. 1 Introduction The unprecedented progress in the development of neural large language models (LLMs; Devlin et al., 2019; Radford et al., 2019; Brown et al., 2020; Zhang et al., 2022) has been accompanied by a comparable proliferation of methods that aim to better understand and characterize models’ linguis- tic capacities (Linzen et al., 2016; Ettinger et al., 2016; Alishahi et al., 2019; Hu et al., 2020; Jeretic et al., 2020; Mueller et al., 2020, i.a.). Of the many methods for this, the minimal-pair paradigm (MPP), Subj. Verb. Agreement ... ... ... ... Space of Candidate Prefixes Matched Mismatched Test Suite: Subject Verb Agreement sample = Island Effects = Filler-gaps = Binding ... = ... acceptable unacceptable = Wikipedia Figure 1: We measure the impact of different contexts on the performance of an LM on linguistic acceptability tasks by prefixing sentences (here, sourced from subject- verb agreement challenge sets) from a diverse sources. Each block represents a sentence: Red striped blocks are unacceptable sentences within a given task, while green solid ones are acceptable. We also vary the diver- sity of prefixes by sampling them from tasks/datasets different from the test suite (indicated by shape). which is methodologically standard in linguistics, has emerged as a popular approach to evaluate mod- els’ knowledge of linguistic phenomena in an unsu- pervised manner (Marvin and Linzen, 2018; Kann et al., …

Content truncated to 500 words