Modeling the Interaction Between Perception-Based and Production-Based Learning in Children’s Early Acquisition of Semantic Knowledge

Proceedings of the 25th Conference on Computational Natural Language Learning (CoNLL), pages 391–407 November 10–11, 2021. ©2021 Association for Computational Linguistics 391 Modeling the Interaction Between Perception-Based and Production-Based Learning in Children’s Early Acquisition of Semantic Knowledge Mitja Nikolaus1,2 mitja.nikolaus@univ-amu.fr Abdellah Fourtassi1 abdellah.fourtassi@univ-amu.fr 1Aix Marseille Univ, Université de Toulon, CNRS, LIS, Marseille, France 2Aix-Marseille Univ, CNRS, LPL, Aix-en-Provence, France Abstract Children learn the meaning of words and sen- tences in their native language at an impres- sive speed and from highly ambiguous input. To account for this learning, previous com- putational modeling has focused mainly on the study of perception-based mechanisms like cross-situational learning. However, children do not learn only by exposure to the input. As soon as they start to talk, they practice their knowledge in social interactions and they re- ceive feedback from their caregivers. In this work, we propose a model integrating both perception- and production-based learning us- ing artificial neural networks which we train on a large corpus of crowd-sourced images with corresponding descriptions. We found that production-based learning improves per- formance above and beyond perception-based learning across a wide range of semantic tasks including both word- and sentence-level se- mantics. In addition, we documented a syn- ergy between these two mechanisms, where their alternation allows the model to converge on more balanced semantic knowledge. The broader impact of this work is to highlight the importance of modeling language learning in the context of social interactions where chil- dren are not only understood as passively ab- sorbing the input, but also as actively partic- ipating in the construction of their linguistic knowledge. 1 Introduction An important aspect of language acquisition is learning how to map linguistic forms to meanings. This involves both mapping individual word forms (e.g., “dog”) to concepts of the world (e.g. the cate- gory DOG) and mapping the relationship between words in a sentence (e.g., “the dog chases the ball”) to a given event configuration in the world (i.e., that the dog is the agent performing the act of chasing on the ball, the semantic patient). Children manage to learn this mapping in their native language at an impressive speed (Fisher and Gleitman, 2002; Golinkoff et al., 2013; Frank et al., 2021) and de- spite the high ambiguity of this task in the natural context where language learning occurs (Quine, 1960). Perception-based learning Much modeling effort has focused on learning from the multimodal input that children perceive around them. These models are based on Cross-Situational Learning (hereafter XSL): While a single word- world mapping situation is ambiguous, being ex- posed to many situations allows the learner to nar- row down, over time, the set of possible associa- tions. This kind of learning has been demonstrated using toy situations in controlled laboratory testing with children (Smith and Yu, 2008). It has also been shown to scale up to more realistic learning contexts using a combination of NLP and computer vision tools (Chrupała et al., 2015; Vong and Lake, 2020, 2021). Production-based learning Learning from …

Content truncated to 500 words