Neural reality of argument structure constructions

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7410 - 7423 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Neural reality of argument structure constructions Bai Li1,4, Zining Zhu1,4 Guillaume Thomas2, Frank Rudzicz1,4,5, Yang Xu1,3,4 1 University of Toronto, Department of Computer Science 2 University of Toronto, Department of Linguistics 3 University of Toronto, Cognitive Science Program 4 Vector Institute for Artificial Intelligence 5 Unity Health Toronto {bai, zining, frank, yangxu}@cs.toronto.edu guillaume.thomas@utoronto.ca Abstract In lexicalist linguistic theories, argument struc- ture is assumed to be predictable from the meaning of verbs. As a result, the verb is the primary determinant of the meaning of a clause. In contrast, construction grammarians propose that argument structure is encoded in constructions (or form-meaning pairs) that are distinct from verbs. Decades of psycholin- guistic research have produced substantial em- pirical evidence in favor of the construction view. Here we adapt several psycholinguistic studies to probe for the existence of argument structure constructions (ASCs) in Transformer- based language models (LMs). First, using a sentence sorting experiment, we find that sen- tences sharing the same construction are closer in embedding space than sentences sharing the same verb. Furthermore, LMs increasingly prefer grouping by construction with more in- put data, mirroring the behaviour of non-native language learners. Second, in a “Jabberwocky” priming-based experiment, we find that LMs associate ASCs with meaning, even in seman- tically nonsensical sentences. Our work of- fers the first evidence for ASCs in LMs and highlights the potential to devise novel prob- ing methods grounded in psycholinguistic re- search. 1 Introduction Pretrained Transformer-based language models (LMs) such as BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019b) have recently achieved impressive results on many natural lan- guage tasks, spawning a new interdisciplinary field of aligning LMs with linguistic theory and prob- ing the linguistic capabilities of LMs (Linzen and Baroni, 2021). Most probing work so far has in- vestigated the linguistic knowledge of LMs on phe- nomena such as agreement, binding, licensing, and movement (Warstadt et al., 2020a; Hu et al., 2020) Transitive Bob cut the bread S V O S acts on O Ditransitive Bob cut Joe the bread S V O1 O2 S transfers O2 to O1 Caused motion Bob cut the bread into the pan S V O Path S causes O to move via Path Resultative Bob cut the bread apart S V O State S causes O to become State Figure 1: Four argument structure constructions (ASCs) used by Bencini and Goldberg (2000), with example sentences (top right). Constructions are map- pings between form (bottom left) and meaning (bottom right). with a particular focus on determining whether a sentence is linguistically acceptable (Schütze, 1996). Relatively little work has attempted to de- termine whether the linguistic knowledge induced by LMs is more similar to a formal grammar of the sort postulated by mainstream generative linguis- tics (Chomsky, 1965, 1981, 1995), or to a network of form-meaning pairs as advocated …

Content truncated to 500 words