Trouble on the Horizon: Forecasting the Derailment of Online Conversations as they Develop

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 4743–4754, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 4743 Trouble on the Horizon: Forecasting the Derailment of Online Conversations as they Develop Jonathan P. Chang Cornell University jpc362@cornell.edu Cristian Danescu-Niculescu-Mizil Cornell University cristian@cs.cornell.edu Abstract Online discussions often derail into toxic ex- changes between participants. Recent efforts mostly focused on detecting antisocial behav- ior after the fact, by analyzing single com- ments in isolation. To provide more timely notice to human moderators, a system needs to preemptively detect that a conversation is heading towards derailment before it actually turns toxic. This means modeling derailment as an emerging property of a conversation rather than as an isolated utterance-level event. Forecasting emerging conversational proper- ties, however, poses several inherent modeling challenges. First, since conversations are dy- namic, a forecasting model needs to capture the flow of the discussion, rather than proper- ties of individual comments. Second, real con- versations have an unknown horizon: they can end or derail at any time; thus a practical fore- casting model needs to assess the risk in an on- line fashion, as the conversation develops. In this work we introduce a conversational fore- casting model that learns an unsupervised rep- resentation of conversational dynamics and ex- ploits it to predict future derailment as the con- versation develops. By applying this model to two new diverse datasets of online conversa- tions with labels for antisocial events, we show that it outperforms state-of-the-art systems at forecasting derailment. 1 Introduction “Ch´e saetta previsa vien pi`u lenta.”1 – Dante Alighieri, Divina Commedia, Paradiso Antisocial behavior is a persistent problem plaguing online conversation platforms; it is both widespread (Duggan, 2014) and potentially dam- aging to mental and emotional health (Raskauskas and Stoltz, 2007; Akbulut et al., 2010). The strain this phenomenon puts on community maintainers 1“The arrow one foresees arrives more gently.” (1) [User A] What does [quote omitted] refer to? I as- sume it should be written from June 2010 to December 2011 and we should precise [sic] the months. (2) [User B] No. It refers to 2007-2011 Belgian political crisis (3) [User A] 2007-2011 Belgian political crisis is a lit- tle bit [of original research]. It merges 2 crisis in 1. Sources 2 and 4 in the article talk about a 18 months cri- sis in 2010-2011, ie what I refer to. What are the reliable sources that make this crisis go back to 2007? (4) [User B] Yes it’s not ridiculous at all to claim [it’s original research] because it doesn’t fit your argument. A crisis can be composed out of several smaller crisis. It’s not original research if some of your sources only talk about parts [...] (5) [User A] Where is the source that claim the crisis is 4 year long? Sources state claim it is 18 month long and refer to the period from June 2010 to December 2011. …

Content truncated to 500 words