“What is on your mind?” Automated Scoring of Mindreading in Childhood and Early Adolescence

Proceedings of the 28th International Conference on Computational Linguistics, pages 6217–6228 Barcelona, Spain (Online), December 8-13, 2020 6217 “What is on your mind?” Automated Scoring of Mindreading in Childhood and Early Adolescence Venelin Kovatchev1, Phillip Smith2, Mark Lee2, Imogen Grumley Traynor1, Irene Luque Aguilera1 and Rory T. Devine1 1 School of Psychology, University of Birmingham 2 School of Computer Science, University of Birmingham 52 Prichatts Road, Edgbaston, B15 2SB, United Kingdom {v.o.kovatchev , P.Smith.7 , m.g.lee , I.GrumleyTraynor , I.LuqueAguilera , R.T.Devine} @bham.ac.uk Abstract In this paper we present the first work on the automated scoring of mindreading ability in middle childhood and early adolescence. We create MIND-CA, a new corpus of 11,311 question-answer pairs in English from 1,066 children aged 7 to 14. We perform machine learning experiments and carry out extensive quantitative and qualitative evaluation. We obtain promising results, demonstrating the applicability of state-of-the-art NLP solutions to a new domain and task. 1 Introduction Successfully navigating day-to-day social interactions often hinges on being able to recognize that others can hold different beliefs, thoughts, ideas, intentions and desires. Popular culture and literature are replete with instances that require us to enter into the mind of someone else in order to explain their actions. For example, the final scene of Shakespeare’s Romeo and Juliet is made all the more poignant because the audience know that Juliet is still alive and that Romeo has killed himself because he had a mistaken belief that she was dead. The audience is able to understand both the state of the world (i.e.: Juliet is still alive) and the mistaken belief of Romeo (i.e.: Juliet is dead). The ability to attribute mental states to others in order to explain or predict behavior, called “theory of mind” or “mindreading” has intrigued researchers in disciplines as diverse as psychology, philoso- phy, neuroscience, economics, and literary theory for four decades (Hughes and Devine, 2015). The long-standing interest in mindreading is not surprising as children’s performance on standard tests of mindreading has real-world significance. Longitudinal studies demonstrate that children who excel at tests of mindreading are more likely than their peers to be identified as popular by classmates (Banerjee et al., 2011), have reciprocated friendships (Fink et al., 2015), and be viewed by their teachers as socially competent (Devine et al., 2016). Furthermore, case-control studies show that deficits in mindreading are more common among individuals with a range of mental health problems and neuro-developmental conditions associated with poorer social functioning (Cotter et al., 2018). Given the prognostic impor- tance of mindreading in children, in this paper we present the joint work of an inter-disciplinary team of psychologists, computer scientists, and linguists on the automatic scoring of mindreading ability in middle childhood and early adolescence. The great majority of studies on mindreading have focused on identifying the age at which preschool children recognize that others have desires, beliefs and knowledge that differ from their own (Wellman and Liu, 2004). However, the past decade has witnessed increased interest in the continued …

Content truncated to 500 words