SLABERT Talk Pretty One Day: Modeling Second Language Acquisition with BERT

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 11763–11777 July 9-14, 2023 ©2023 Association for Computational Linguistics SLABERT Talk Pretty One Day: Modeling Second Language Acquisition with BERT ⋆Aditya Yadavalli1 ⋆Alekhya Yadavalli2 Vera Tobin2 1 Karya Inc.∗ 2 Language and Cognition Lab, Case Western Reserve University aditya@karya.in {alekhya.yadavalli, vera.tobin}@case.edu Abstract Second language acquisition (SLA) research has extensively studied cross-linguistic trans- fer, the influence of linguistic structure of a speaker’s native language [L1] on the success- ful acquisition of a foreign language [L2]. Ef- fects of such transfer can be positive (facilitat- ing acquisition) or negative (impeding acqui- sition). We find that NLP literature has not given enough attention to the phenomenon of negative transfer. To understand patterns of both positive and negative transfer between L1 and L2, we model sequential second lan- guage acquisition in LMs. Further, we build a Mutlilingual Age Ordered CHILDES (MAO- CHILDES)—a dataset consisting of 5 typolog- ically diverse languages, i.e., German, French, Polish, Indonesian, and Japanese—to under- stand the degree to which native Child-Directed Speech (CDS) [L1] can help or conflict with En- glish language acquisition [L2]. To examine the impact of native CDS, we use the TILT-based cross lingual transfer learning approach estab- lished by Papadimitriou and Jurafsky (2020) and find that, as in human SLA, language fam- ily distance predicts more negative transfer. Ad- ditionally, we find that conversational speech data shows greater facilitation for language ac- quisition than scripted speech data. Our find- ings call for further research using our novel Transformer-based SLA models and we would like to encourage it by releasing our code, data, and models. 1 Introduction Cross-linguistic transfer can be described as the influence of native language [L1] properties on a speaker’s linguistic performance in a new, for- eign language [L2]. The interaction of the linguis- tic structure of a speaker’s L1 with the successful ∗ Work done as a visiting researcher at Case Western Reserve University Code, data and models are here: https://github.com/ AdityaYadavalli1/SLABERT ⋆ Authors contributed equally acquisition of L2 results in what are termed as transfer effects. Transfer effects appear in vari- ous aspects of linguistic performance, including vocabulary, pronunciation, and grammar (Jarvis and Pavlenko, 2007). Cross-linguistic transfer can be positive or negative in nature: positive transfer refers to the facilitating effects of one language in acquiring another (e.g., of Spanish vocabulary in ac- quiring French) and negative transfer between the learner’s native [L1] and target [L2] languages, pro- ducing errors. The greater the differences between two languages, the greater the negative effects. While cross-lingual transfer has received consid- erable attention in NLP research (Wu and Dredze, 2019; Wu et al., 2019; Conneau et al., 2017, 2018; Artetxe et al., 2018; Ruder et al., 2017), most of this research has concentrated on practical impli- cations such as the degree to which the right to- kenizer can optimize cross-lingual transfer, and has not looked at the kind of sequential transfer relationships that arise in human second language …

Content truncated to 500 words