Learned Construction Grammars Converge Across Registers Given Increased Exposure

Proceedings of the 25th Conference on Computational Natural Language Learning (CoNLL), pages 268–278 November 10–11, 2021. ©2021 Association for Computational Linguistics 268 Learned Construction Grammars Converge Across Registers Given Increased Exposure Jonathan Dunn University of Canterbury Christchurch, NZ first.last@canterbury.ac.nz Harish Tayyar Madabushi University of Sheffield Sheffield, UK f.last@sheffield.ac.uk Abstract This paper measures the impact of increased exposure on whether learned construction grammars converge onto shared representa- tions when trained on data from different regis- ters. Register influences the frequency of con- structions, with some structures common in formal but not informal usage. We expect that a grammar induction algorithm exposed to dif- ferent registers will acquire different construc- tions. To what degree does increased exposure lead to the convergence of register-specific grammars? The experiments in this paper sim- ulate language learning in 12 languages (half Germanic and half Romance) with corpora rep- resenting three registers (Twitter, Wikipedia, Web). These simulations are repeated with in- creasing amounts of exposure, from 100k to 2 million words, to measure the impact of ex- posure on the convergence of grammars. The results show that increased exposure does lead to converging grammars across all languages. In addition, a shared core of register-universal constructions remains constant across increas- ing amounts of exposure. 1 Exposure and Convergence The central question that this work aims to an- swer is whether register-specific grammars con- verge onto shared representations when exposed to more training data. Variation in the context of pro- duction, called register variation, has a significant impact on the frequency of constructions. For ex- ample, imperative and WH-question constructions are much more frequent in informal or conversa- tional speech, while declarative constructions are much more frequent in formal written usage (Fodor and Crowther, 2002; Sampson, 2002). At the same time, usage-based grammar views language as a complex adaptive system that emerges given exposure to usage (Bybee, 2006; Beckner et al., 2009). Thus, a language learner is expected to be strongly influenced by the ob- served frequency of constructions. Given this wide variance in the frequency of constructions across registers, it is conceivable that learners exposed to different registers learn different constructions. This paper simulates the language acquisition process from a usage-based perspective by learning Construction Grammars, called CxGs (Goldberg, 1995, 2006; Langacker, 2008). A constructional approach to language focuses on symbolic form- meaning mappings that are potentially idiomatic. Previous work on computational CxG has explored how to discover potential constructions (Wible and Tsao, 2010; Forsberg et al., 2014; Dunn, 2017), the process of construction learning (Barak and Goldberg, 2017; Barak et al., 2017), and whether constructional information is implicitly encoded in models like BERT (Tayyar Madabushi et al., 2020). A commonly discussed example of a construc- tion is the ditransitive in (a1). CxGs use a constraint-based formalism in which each slot in the construction is defined by a particular slot- constraint; in (a1) these are syntactic constraints. One of the important ideas in CxG is that construc- tions themselves carry a meaning. For example, the ditransitive construction carries …

Content truncated to 500 words