How to disagree well: Investigating the dispute tactics used on Wikipedia

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3824 - 3837 December 7-11, 2022 ©2022 Association for Computational Linguistics How to disagree well: Investigating the dispute tactics used on Wikipedia Christine de Kock Department of Computer Science and Technology University of Cambridge cd700@cam.ac.uk Tom Stafford Department of Psychology University of Sheffield t.stafford@sheffield.ac.uk Andreas Vlachos Department of Computer Science and Technology University of Cambridge av308@cam.ac.uk Abstract Disagreements are frequently studied from the perspective of either detecting toxicity or analysing argument structure. We propose a framework of dispute tactics which unifies these two perspectives, as well as other dia- logue acts which play a role in resolving dis- putes, such as asking questions and providing clarification. This framework includes a pref- erential ordering among rebuttal-type tactics, ranging from ad hominem attacks to refuting the central argument. Using this framework, we annotate 213 disagreements (3,865 utter- ances) from Wikipedia Talk pages. This allows us to investigate research questions around the tactics used in disagreements; for instance, we provide empirical validation of the approach to disagreement recommended by Wikipedia. We develop models for multilabel prediction of dispute tactics in an utterance, achieving the best performance with a transformer-based la- bel powerset model. Adding an auxiliary task to incorporate the ordering of rebuttal tactics further yields a statistically significant increase. Finally, we show that these annotations can be used to provide useful additional signals to im- prove performance on the task of predicting escalation. 1 Introduction Disagreements are pervasive in online commu- nication. While usually perceived as a negative phenomenon, research in psychology has shown that debate and disagreement can promote beliefs that are better supported by evidence (Landemore, 2012; Mercier, 2018). Hallsson and Kappel (2020) argue that, in the ideal case, a range of arguments are considered for both sides, increasing the partic- ipants’ ability to find good arguments in defence of a view and to critique reasons against it, and thereby forcing one to interact with reasoning and evidence which may otherwise be discarded due to confirmation bias. Prior work in NLP has investigated detecting negative artefacts of online disagreements such as personal attacks or hate speech, e.g. Wulczyn et al. (2017) and Waseem and Hovy (2016). Research in argumentation mining instead looks at identifying argument structures (Lawrence and Reed, 2020) and inferring the quality of arguments (Habernal and Gurevych, 2016), often focusing on classical theories of argumentation (e.g. Aristotelian) which do not include less desirable aspects such as per- sonal attacks. However, real world disagreements often contain both well-structured arguments and attacks, in addition to other dialogue acts, such as asking for and providing clarification. In this work, we propose a framework of dis- pute tactics consisting of rebuttal and coordina- tion strategies, denoting the role of a particular utterance in the context of a disagreement discus- sion. We build on the disagreement hierarchy pro- posed by Graham (2008), which includes a pref- erential ordering between different rebuttal tactics. We introduce WikiTactics1: a set of 213 disputes …

Content truncated to 500 words