Autoregressive Affective Language Forecasting: A Self-Supervised Task

Proceedings of the 28th International Conference on Computational Linguistics, pages 2913–2923 Barcelona, Spain (Online), December 8-13, 2020 2913 Autoregressive Affective Language Forecasting: A Self-Supervised Task Matthew Matero Stony Brook University mmatero@cs.stonybrook.edu H. Andrew Schwartz Stony Brook University has@cs.stonybrook.edu Abstract Human natural language is mentioned at a specific point in time while human emotions change over time. While much work has established a strong link between language use and emotional states, few have attempted to model emotional language in time. Here, we introduce the task of affective language forecasting – predicting future change in language based on past changes of language, a task with real-world applications such as treating mental health or forecasting trends in consumer confidence. We establish some of the fundamental autoregressive characteristics of the task (necessary history size, static versus dynamic length, varying time-step resolutions) and then build on popular sequence models for words to instead model sequences of language-based emotion in time. Over a novel Twitter dataset of 1,900 users and weekly + daily scores for 6 emotions and 2 additional linguistic attributes, we find a novel dual-sequence GRU model with decayed hidden states achieves best results (r = .66). We make our anonymized dataset as well as task setup and evaluation code available for others to build on. 1 Introduction With the growth of social media, natural language processing has increasingly turned toward problems in the social scientific domains. Language has been used to predict personality disorders, political ideology, and mental health (Preotiuc-Pietro et al., 2016a; Iyyer et al., 2014; Coppersmith et al., 2014). However, such predictions are typically made cross-documents or -users rather than cross-time (forecasting the future). Predictions across time can enable another level of applications for NLP in the social sciences. For example, knowing how someone’s mood may change next week can be a valuable insight into preemp- tively preparing mental health assistance. We may be able to better predict a substance abuse relapse or suicide attempt, or provide individual insights into the precipitators of such events.1 Applications also exist in other domains, such as correlating mood and personal buying habits, or predicting the mood of performers (e.g. professional athletes) as insight into how well they will perform next. One of the challenges for doing temporal predictions based on language is the availability of time- series data. Most user-level datasets only provide one measurement, typically of a demographic or psychological trait (something that doesn’t change frequently). On the other hand, document-level data (such as that of most sentiment analysis) may provide information across time, but they are typically divorced from the individual. Alternatively, instead of relying on annotations or self-reported data, one can simply predict the future language-based measurement (i.e. as output from another model). Here, we propose a method of forecasting any future dimension of language, based on their current and past language. Formally, we define affective language forecasting as finding a function of past language (f) which estimates the output of a function of future language (a): f(Xd t−1, Xd t−2, …

Content truncated to 500 words