GPT-D: Inducing Dementia-related Linguistic Anomalies by Deliberate Degradation of Artificial Neural Language Models

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 1866 - 1877 May 22-27, 2022 c⃝2022 Association for Computational Linguistics GPT-D: Inducing Dementia-related Linguistic Anomalies by Deliberate Degradation of Artificial Neural Language Models Changye Li1, David Knopman2, Weizhe Xu3, Trevor Cohen3, and Serguei Pakhomov4 1Institute of Health Informatics, University of Minnesota 2Mayo Clinic, Rochester, MN 3Biomedical and Health Informatics, University of Washington 4Pharmaceutical Care and Health Systems, University of Minnesota 1, 4{lixx3013, pakh0002}@umn.edu 2{knopman}@mayo.edu 3{xuweizhe, cohenta}@uw.edu Abstract Deep learning (DL) techniques involving fine- tuning large numbers of model parameters have delivered impressive performance on the task of discriminating between language produced by cognitively healthy individuals, and those with Alzheimer’s disease (AD). However, ques- tions remain about their ability to generalize beyond the small reference sets that are pub- licly available for research. As an alternative to fitting model parameters directly, we pro- pose a novel method by which a Transformer DL model (GPT-2) pre-trained on general En- glish text is paired with an artificially degraded version of itself (GPT-D), to compute the ratio between these two models’ perplexities on lan- guage from cognitively healthy and impaired in- dividuals. This technique approaches state-of- the-art performance on text data from a widely used "Cookie Theft" picture description task, and unlike established alternatives also gener- alizes well to spontaneous conversations. Fur- thermore, GPT-D generates text with character- istics known to be associated with AD, demon- strating the induction of dementia-related lin- guistic anomalies. Our study is a step toward better understanding of the relationships be- tween the inner workings of generative neural language models, the language that they pro- duce, and the deleterious effects of dementia on human speech and language characteristics. 1 Introduction Alzheimer’s disease (AD) dementia affects every aspect of cognition, including language use. Over 50 million people are currently diagnosed with AD dementia, and this number is expected to triple by 2050 (Organization et al., 2017; Patterson, 2018; Prince et al., 2016). Furthermore, over half of the individuals living with dementia are undiagnosed (Lang et al., 2017). While AD has no known cure, timely diagnosis can prevent or alleviate adverse outcomes ranging from anxiety over unexplained symptoms to family discord and catastrophic events (Stokes et al., 2015; Boise et al., 1999; Bond et al., 2005). However, diagnosis of AD dementia is time-consuming and challenging for patients and physicians alike, and currently relies on patient and caregiver reports, extensive neuropsychological ex- aminations, and invasive imaging and diagnostic procedures (Patterson, 2018). Automated analysis of spoken language can potentially provide accu- rate, easy-to-use, safe and cost-effective tools for monitoring AD-related cognitive markers. In par- ticular, studies have demonstrated that supervised machine learning methods can learn to differenti- ate accurately between patients with dementia and healthy controls (Fraser et al., 2016; Orimaye et al., 2017), with particularly strong performance from recent deep learning (DL) models (Balagopalan et al., 2020; Roshanzamir et al., 2021). However, the large number of parameters employed in DL presents a danger of …

Content truncated to 500 words