Survival text regression for time-to-event prediction in conversations

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1219–1229 August 1–6, 2021. ©2021 Association for Computational Linguistics 1219 Survival text regression for time-to-event prediction in conversations Christine De Kock University of Cambridge Department of Computer Science cd700@cam.ac.uk Andreas Vlachos University of Cambridge Department of Computer Science av308@cam.ac.uk Abstract Time-to-event prediction tasks are common in conversation modelling, for applications such as predicting the length of a conversation or when a user will stop contributing to a plat- form. Despite the fact that it is natural to frame such predictions as regression tasks, re- cent work has modelled them as classification tasks, determining whether the time-to-event is greater than a pre-determined cut-off point. While this allows for the application of classi- fication models which are well studied in NLP, it imposes a formulation that is contrived, as well as less informative. In this paper, we ex- plore how to handle time-to-event forecasting in conversations as regression tasks. We focus on a family of regression techniques known as survival regression, which are commonly used in the context of healthcare and reliabil- ity engineering. We adapt these models to time-to-event prediction in conversations, us- ing linguistic markers as features. On three datasets, we demonstrate that they outperform commonly considered text regression methods and comparable classification models. 1 Introduction The task of predicting when an event will occur in a conversation frequently arises in NLP research. For instance, Backstrom et al. (2013) and Zhang et al. (2018b) predict when a conversation thread will terminate. Danescu-Niculescu-Mizil et al. (2013) define the task of forecasting when users will cease to interact on a social network based on their lan- guage use. Although these questions naturally lend themselves to regression, this presents some dif- ficulties: datasets may be highly skewed towards shorter durations (Zhang et al., 2018b) and samples with a longer duration can contribute inordinately to error terms during training. Furthermore, clas- sical regression models do not explicitly consider the effect of time as distinct from other features. The abovementioned studies instead frame the time-to-event prediction as a classification task, pre- dicting whether the current state will continue for a set number of additional timesteps. For instance, Backstrom et al. (2013) predict whether the number of responses in a thread will exceed 8, after seeing 5 utterances. This presents obvious limitations; such a setup would assign the same error for mistakenly classifying conversations of respectively 9 and 30 utterances as “short”. Additionally, its predictions are less informative: predicting that a conversation will be more than 8 utterances long is less telling than predicting whether it will be 9 or 30. In this paper, we propose that survival regres- sion is a more appropriate modelling framework for predicting when an event will occur in a con- versation. Survival regression aims to predict the probability of an event of interest at different points in time, taking into account features of a subject as seen up to the prediction time. We apply sur- vival models to two tasks: …

Content truncated to 500 words