Creation and evaluation of timelines for longitudinal user posts

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3791–3804 May 2-6, 2023 ©2023 Association for Computational Linguistics Creation and evaluation of timelines for longitudinal user posts Anthony Hills1, Adam Tsakalidis1,2, Federico Nanni2, Ioannis Zachos3, Maria Liakata1,2,4 1Queen Mary University of London, 2The Alan Turing Institute, 3University of Cambridge, 4University of Warwick {a.r.hills;a.tsakalidis;m.liakata}@qmul.ac.uk Abstract There is increasing interest to work with user generated content in social media, especially textual posts over time. Currently there is no consistent way of segmenting user posts into timelines in a meaningful way that improves the quality and cost of manual annotation. Here we propose a set of methods for segmenting lon- gitudinal user posts into timelines likely to con- tain interesting moments of change in a user’s behaviour, based on their online posting activ- ity. We also propose a novel framework for evaluating timelines and show its applicabil- ity in the context of two different social media datasets. Finally, we present a discussion of the linguistic content of highly ranked timelines. 1 1 Introduction An increasing body of work considers time-aware models trained on social media data for a number of different tasks, including personal event identifi- cation (Li and Cardie, 2014; Li et al., 2014; Chang et al., 2016a), suicidal ideation and suicide risk de- tection (Coppersmith et al., 2014, 2018; Cao et al., 2019; Matero et al., 2019; Sawhney et al., 2020, 2021). For such tasks deriving meaningful time- lines (i.e. sequences of posts by individuals), con- taining examples of the phenomenon under study from large-scale collections, together with asso- ciated annotations, is crucial. This is especially important for computational approaches in mental health (MH) given the surging numbers of those seeking help online (Neary and Schueller, 2018). Earlier work on personal life event detection considered selecting salient timelines through topic modelling (Li and Cardie, 2014; Li et al., 2014) or through a non-parametric generative ap- proach (Chang et al., 2016a). However, such ap- proaches are unsuitable for identifying changes in mood or MH more generally. Specifically, since 1https://github.com/Maria-Liakata-NLP-Group/ timeline_selection_and_evaluation timelines are selected based on linguistic content this introduces a sampling bias for downstream linguistic analysis and annotation (Olteanu et al., 2019; Mishra et al., 2019). In recent work on suici- dal ideation detection, timelines are chosen as the N most recent posts (Sawhney et al., 2020), which are not necessarily the most salient for annotation. Present Work: We propose a set of methods and associated evaluation framework for identifying salient timelines from the history of social me- dia users to be annotated for changes in a user’s behaviour, as revealed through their textual data. Applying our methods in the domain of MH, we follow earlier work in hypothesising that posting behaviour can be a proxy for changes in the MH of an individual (De Choudhury et al., 2016). There- fore we develop methods for creating timelines based on time-series of posting frequency, such as change-point and anomaly detection approaches, and evaluate these against keyword-based …

Content truncated to 500 words