Improving the Generalizability of Depression Detection by Leveraging Clinical Questionnaires

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 8446 - 8459 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Improving the Generalizability of Depression Detection by Leveraging Clinical Questionnaires Thong Nguyen1,2, Andrew Yates1,2, Ayah Zirikly3, Bart Desmet4, and Arman Cohan5 1University of Amsterdam, Amsterdam, Netherlands 2Max Planck Institute for Informatics, Saarbrücken, Germany 3Johns Hopkins University, Baltimore, Maryland 4National Institutes of Health, Bethesda, Maryland 5Allen Institute for AI, Seattle, WA Abstract Automated methods have been widely used to identify and analyze mental health conditions (e.g., depression) from various sources of in- formation, including social media. Yet, deploy- ment of such models in real-world healthcare applications faces challenges including poor out-of-domain generalization and lack of trust in black box models. In this work, we pro- pose approaches for depression detection that are constrained to different degrees by the pres- ence of symptoms described in PHQ9, a ques- tionnaire used by clinicians in the depression screening process. In dataset-transfer experi- ments on three social media datasets, we find that grounding the model in PHQ9’s symptoms substantially improves its ability to generalize to out-of-distribution data compared to a stan- dard BERT-based approach. Furthermore, this approach can still perform competitively on in- domain data. These results and our qualitative analyses suggest that grounding model predic- tions in clinically-relevant symptoms can im- prove generalizability while producing a model that is easier to inspect. 1 Introduction Given the significance of mental health as a pub- lic health challenge (Brådvik, 2018), much work has investigated approaches for detecting mental health conditions using social media text (Yates et al., 2017; Coppersmith et al., 2018; Shing et al., 2020; Harrigian et al., 2021). Such approaches could be used by at-risk users and their clinicians to monitor behavioral changes (e.g., by monitor- ing changes in the presence of symptoms related to depression as treatment progresses. These ap- proaches generally rely on datasets consisting of users with self-reported diagnoses (e.g., based on a statement like “I was just diagnosed with de- pression”) for training and evaluation (e.g., Yates et al., 2017; Cohan et al., 2018). Despite promis- ing results on these tasks, related work argues that assessing depression and suicidal behavior is dif- ficult in practical settings and even experienced clinicians frequently struggle to correctly interpret signals (Coppersmith et al., 2018). Furthermore, recent work has found that models trained on par- ticular mental health datasets do not always gen- eralize to others. Harrigian et al. (2020); Ernala et al. (2019) find that systematic, spurious differ- ences between diagnosed and control users can prevent trained models from generalizing to even other, similar social media data. Similarly, outside the mental health domain, recent work reports that neural models often struggle to generalize to data outside their training distribution (Geirhos et al., 2020; D’Amour et al., 2020; Harrigian et al., 2020). In this work, we explore approaches for con- straining the behavior of depression detection meth- ods by the presence of symptoms known to be re- …

Content truncated to 500 words