Fair and Argumentative Language Modeling for Computational Argumentation

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7841 - 7861 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Fair and Argumentative Language Modeling for Computational Argumentation Carolin Holtermann1, Anne Lauscher2, Simone Paolo Ponzetto1 1Data and Web Science Group, University of Mannheim, Germany 2MilaNLP, Bocconi University, Italy cholterm@mail.uni-mannheim.de anne.lauscher@unibocconi.it simone@informatik.uni-mannheim.de Abstract Although much work in NLP has focused on measuring and mitigating stereotypical bias in semantic spaces, research addressing bias in computational argumentation is still in its infancy. In this paper, we address this re- search gap and conduct a thorough investiga- tion of bias in argumentative language mod- els. To this end, we introduce AB B A , a novel resource for bias measurement specifi- cally tailored to argumentation. We employ our resource to assess the effect of argumen- tative fine-tuning and debiasing on the intrin- sic bias found in transformer-based language models using a lightweight adapter-based ap- proach that is more sustainable and parameter- efficient than full fine-tuning. Finally, we ana- lyze the potential impact of language model de- biasing on the performance in argument qual- ity prediction, a downstream task of compu- tational argumentation. Our results show that we are able to successfully and sustainably re- move bias in general and argumentative lan- guage models while preserving (and some- times improving) model performance in down- stream tasks. We make all experimental code and data available at https://github.com/ umanlp/FairArgumentativeLM. 1 Introduction Recently, pre-trained language models (PLMs), e.g., BERT (Devlin et al., 2019), RoBERTa (Liu et al., 2019), GPT-2 (Radford et al., 2019) and DialoGPT (Zhang et al., 2020) have been shown to encode and amplify a range of stereotypical biases, such as racism, and sexism (e.g., Kurita et al., 2019a; Dev et al., 2020; Nangia et al., 2020; Lauscher et al., 2021a, inter alia). While such types of biases provide the basis for interesting academic research, e.g., historical analyses (e.g., Garg et al., 2018; Tripodi et al., 2019; Walter et al., 2021, inter alia), stereotyping constitutes a representational harm (Barocas et al., 2017; Blodgett et al., 2020), and can lead in many concrete socio-technical ap- plication scenarios to severe ethical issues by re- inforcing societal biases (Hovy and Spruit, 2016; Shah et al., 2020; Mehrabi et al., 2021). But while prior work has focused on how to eval- uate and mitigate unfair biases for general-purpose LMs (e.g., Webster et al., 2020) and their appli- cations to specific domains and genre like, for in- stance, conversational LMs (e.g., Barikeri et al., 2021), there has been little attention to the problem of bias in argumentative language. This is despite previous work from Spliethöver and Wachsmuth (2020) pointing out the high potential for harm, due to the high sensitivity of envisioned applications like self-determined opinion formation systems, as well as, crucially, showing that argumentative corpora like those from the online debate portal debate.org (Durmus and Cardie, 2019) do en- code unfair biases, which are likely to be captured by argumentative LMs. This …

Content truncated to 500 words