I Beg to Differ: A study of constructive disagreement in online conversations

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 2017–2027 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 2017 I Beg to Differ: A study of constructive disagreement in online conversations Christine De Kock and Andreas Vlachos University of Cambridge Department of Computer Science and Technology {cd700,av308}@cam.ac.uk Abstract Disagreements are pervasive in human com- munication. In this paper we investigate what makes disagreement constructive. To this end, we construct WikiDisputes, a corpus of 7 425 Wikipedia Talk page conversations that con- tain content disputes, and define the task of predicting whether disagreements will be esca- lated to mediation by a moderator. We evalu- ate feature-based models with linguistic mark- ers from previous work, and demonstrate that their performance is improved by using fea- tures that capture changes in linguistic markers throughout the conversations, as opposed to av- eraged values. We develop a variety of neural models and show that taking into account the structure of the conversation improves predic- tive accuracy, exceeding that of feature-based models. We assess our best neural model in terms of both predictive accuracy and uncer- tainty by evaluating its behaviour when it is only exposed to the beginning of the conver- sation, finding that model accuracy improves and uncertainty reduces as models are exposed to more information. 1 Introduction Disagreements online are a familiar occurrence for any internet user. While they are often per- ceived as a negative phenomenon, disagreements can be useful; as is illustrated in Figure 1, dis- agreements can lead to an improved understanding of a topic by introducing and evaluating different perspectives. Research on online disagreements has focused on mostly negative aspects such as trolling (Cheng et al., 2017), hate speech (Waseem and Hovy, 2016), harassment (Yin et al., 2009) and personal attacks (Wulczyn et al., 2017). Re- cent works by Zhang et al. (2018) and Chang and Danescu-Niculescu-Mizil (2019) study conversa- tions on Wikipedia talk pages that start out as civil but derail into personal attacks, using linguis- Figure 1: An example of a dispute on Wikipedia, show- ing the dispute tag on the article, talk page posts, and summaries of edits occurring during the discussion. tic markers and deep learning approaches, respec- tively. An alternative approach is to study good faith disagreement through debates on online platforms such as ChangeMyView (Tan et al., 2016), de- bate.org (Durmus and Cardie, 2019) and Kialo (Boschi et al., 2019), or formal Oxford-style de- bates (Zhang et al., 2016a). While this has benefits, such as readily available annotation of stances and indication of winning and losing sides, it does not mirror the way people naturally converse. For ex- ample, in formal debates, temporal and structural 2018 constraints are imposed. Moreover, in Oxford-style debates, participants are not motivated to come to a consensus but rather to persuade an audience of a predefined stance (Zhang et al., 2016a). Finally, the task of detecting disagreement in conversations has been studied by a number of authors, e.g. Wang …

Content truncated to 500 words