Hierarchical Modeling for User Personality Prediction: The Role of Message-Level Attention

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5306–5316 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 5306 Hierarchical Modeling for User Personality Prediction: The Role of Message-Level Attention Veronica E. Lynn, Niranjan Balasubramanian, and H. Andrew Schwartz Stony Brook University {velynn, niranjan, has}@cs.stonybrook.edu Abstract Not all documents are equally important. Lan- guage processing is increasingly finding use as a supplement for questionnaires to assess psychological attributes of consenting individ- uals, but most approaches neglect to consider whether all documents of an individual are equally informative. In this paper, we present a novel model that uses message-level attention to learn the relative weight of users’ social me- dia posts for assessing their five factor person- ality traits. We demonstrate that models with message-level attention outperform those with word-level attention, and ultimately yield state- of-the-art accuracies for all five traits by using both word and message attention in combina- tion with past approaches (an average increase in Pearson r of 2.5%). In addition, examina- tion of the high-signal posts identified by our model provides insight into the relationship be- tween language and personality, helping to in- form future work. 1 Introduction Most language-based methods for human attribute prediction assume all documents generated by a person are equally informative. However, this is not necessarily true. Figure 1 gives examples of high and low signal messages for predicting extraversion — one’s tendency to be energized by social inter- action. The high signal messages contain words relating to social interaction (hangin out, chillin), whereas the low signal messages, while still con- taining social-related words, have little clear rele- vance to extraversion. The former examples would ideally be weighted higher by a personality predic- tion model than the latter. This paper applies the idea of modeling docu- ment relevance to the task of personality prediction. Inferring an individual’s personality traits is a fun- damental task in psychology (McCrae and Costa Jr, (a) High signal messages (b) Low signal messages Figure 1: Examples of high and low signal mes- sages identified by our proposed model for predict- ing extraversion. All examples are from the same highly-extroverted user. Shading indicates strength of message-level (blue) and word-level (green) attention. 1997; Mischel et al., 2007), with social scientific applications ranging from public health (Fried- man and Kern, 2014) and marketing (Matz et al., 2017) to personalized medicine (Chapman et al., 2011), mental health care (Bagby et al., 1995), and even providing useful information for downstream NLP tasks (Preot¸iuc-Pietro et al., 2015; Lynn et al., 2017). Recently, researchers from both NLP and psychology have turned toward more accurately assessing personality and other human attributes via language (Mairesse et al., 2007; Schwartz et al., 2013; Park et al., 2015; Kulkarni et al., 2018). The idea behind “language-based assessments” (Park et al., 2015) is that language use patterns can sup- plement and, in part, replace traditional and expen- sive questionnaire-based human assessments. Here, we present a hierarchical neural sequence model over both the words and …

Content truncated to 500 words