Distributional semantic models for the evaluation of disordered language

Proceedings of NAACL-HLT 2013, pages 709–714, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics Distributional semantic models for the evaluation of disordered language Masoud Rouhizadeh†, Emily Prud’hommeaux◦, Brian Roark†, Jan van Santen† †Center for Spoken Language Understanding, Oregon Health & Science University ◦Center for Language Sciences, University of Rochester {rouhizad,vansantj}@ohsu.edu, {emilypx,roarkbr}@gmail.com Abstract Atypical semantic and pragmatic expression is frequently reported in the language of children with autism. Although this atypicality often manifests itself in the use of unusual or un- expected words and phrases, the rate of use of such unexpected words is rarely directly measured or quantified. In this paper, we use distributional semantic models to automat- ically identify unexpected words in narrative retellings by children with autism. The classi- fication of unexpected words is sufficiently ac- curate to distinguish the retellings of children with autism from those with typical develop- ment. These techniques demonstrate the po- tential of applying automated language anal- ysis techniques to clinically elicited language data for diagnostic purposes. 1 Introduction Autism spectrum disorder (ASD) is a neurodevelop- mental disorder characterized by impaired commu- nication and social behavior. Although the symp- toms of ASD are numerous and varied, atypical and idiosyncratic language has been one of the core symptoms observed in verbal individuals with autism since Kanner first assigned a name to the disorder (Kanner, 1943). Atypical language cur- rently serves as a diagnostic criterion in many of the most widely used diagnostic instruments for ASD (Lord et al., 2002; Rutter et al., 2003), and the phe- nomenon is especially marked in the areas of seman- tics and pragmatics (Tager-Flusberg, 2001; Volden and Lord, 1991). Because structured language assessment tools are not always sensitive to the particular atypical seman- tic and pragmatic expression associated with ASD, measures of atypical language are often drawn from spontaneous language samples. Expert manual an- notation and analysis of spontaneous language in young people with ASD has revealed that children and young adults with autism include significantly more bizarre and irrelevant content (Loveland et al., 1990; Losh and Capps, 2003) in their narratives and more abrupt topic changes (Lam et al., 2012) in their conversations than their language-matched typ- ically developing peers. Most normed clinical in- struments for analyzing children’s spontaneous lan- guage, however, focus on syntactic measures and developmental milestones related to the acquisition of vocabulary and syntactic structures. Measures of semantic and pragmatic atypicality in spontaneous language are rarely directly measured. Instead, the degree of language atypicality is often determined via subjective parental reports (e.g., asking a par- ent whether their child has ever used odd phrases (Rutter et al., 2003)) or general impressions dur- ing clinical examination (e.g., rating the child’s de- gree of “stereotyped or idiosyncratic use of words or phrases” on a four-point scale (Lord et al., 2002)). This has led to a lack of reliable and objective infor- mation about the frequency of atypical language use and its precise nature in ASD. In this study, we attempt to automatically detect instances of …

Content truncated to 500 words