Inferring Social Media Users’ Mental Health Status from Multimodal Information

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 6292–6299 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 6292 Inferring Social Media Users’ Mental Health Status from Multimodal Information Zhentao Xu, Ver´onica P´erez-Rosas, Rada Mihalcea University of Michigan Ann Arbor MI, USA {frankxu,vrncapr,mihalcea}@umich.edu Abstract Worldwide, an increasing number of people are suffering from mental health disorders such as depression and anxiety. In the United States alone, one in every four adults suffers from a mental health condition, which makes mental health a pressing concern. In this paper, we explore the use of multimodal cues present in social media posts to predict users’ mental health status. Specifically, we focus on identifying social media activity that either indicates a mental health condition or its onset. We collect posts from Flickr and apply a multimodal approach that consists of jointly analyzing language, visual, and metadata cues and their relation to mental health. We conduct several classification experiments aiming to discriminate between (1) healthy users and users affected by a mental health illness; and (2) healthy users and users prone to mental illness. Our experimental results indicate that using multiple modalities can improve the performance of this classification task as compared to the use of one modality at a time, and can provide important cues into a user’s mental status. Keywords: mental health, social media, multimodal analysis, machine learning 1. Introduction Mental health is a pressing health concern worldwide. Ac- cording to WHO Composite International Diagnostic In- terview (WMH-CIDI), in the United States alone, around one in every four people has experienced a mental illness at some point in their lives (Demyttenaere et al., 2004). Studies addressing mental health in the clinical domain have largely relied on patient’s self-reported experiences, behaviors reported by relatives or friends, and a mental sta- tus examination via psychology testing. Recently, several studies have highlighted the potential to learn about peo- ple’s health and well-being through linguistic and behav- ioral analyses of their social media activity (Burke et al., 2010; Moorhead et al., 2013). The use of social media as a data source provides a less intrusive and more scalable ap- proach to observe and analyze a wide range of self-reported behaviors, thus allowing to capture an individual’s thinking, personalities, and more importantly, mental health status. In addition, with the availability of mobile devices and in- creasing internet speed, more people are willing to share their mood and daily activities on social networks, hence making it possible to obtain high-quality self-reported data around mental health concerns, such as depression, self- harm activity, or eating disorders. In this work, we analyze users’ behavior on the Flickr plat- form, a popular online community where users share per- sonal photographs. Our choice of using Flickr as a data source is motivated by its high-levels of user activity and the multimodal nature of their posts.1 Flickr posts consist of an image and its metadata along with an associated caption and tags, which makes them …

Content truncated to 500 words