From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 11737–11762 July 9-14, 2023 ©2023 Association for Computational Linguistics From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models Shangbin Feng1 Chan Young Park2 Yuhan Liu3 Yulia Tsvetkov1 1University of Washington 2Carnegie Mellon University 3Xi’an Jiaotong University {shangbin, yuliats}@cs.washington.edu chanyoun@cs.cmu.edu lyh6560@stu.xjtu.edu.cn Abstract Language models (LMs) are pretrained on di- verse data sources, including news, discussion forums, books, and online encyclopedias. A sig- nificant portion of this data includes opinions and perspectives which, on one hand, celebrate democracy and diversity of ideas, and on the other hand are inherently socially biased. Our work develops new methods to (1) measure po- litical biases in LMs trained on such corpora, along social and economic axes, and (2) mea- sure the fairness of downstream NLP models trained on top of politically biased LMs. We focus on hate speech and misinformation de- tection, aiming to empirically quantify the ef- fects of political (social, economic) biases in pretraining data on the fairness of high-stakes social-oriented tasks. Our findings reveal that pretrained LMs do have political leanings that reinforce the polarization present in pretrain- ing corpora, propagating social biases into hate speech predictions and misinformation detec- tors. We discuss the implications of our findings for NLP research and propose future directions to mitigate unfairness. 1 Warning: This paper contains examples of hate speech. 1 Introduction Digital and social media have become a major source of political news dissemination (Hermida et al., 2012; Kümpel et al., 2015; Hermida, 2016) with unprecedentedly high user engagement rates (Mustafaraj and Metaxas, 2011; Velasquez, 2012; Garimella et al., 2018). The volume of online discourse surrounding polarizing issues—climate change, gun control, abortion, wage gaps, death penalty, taxes, same-sex marriage, and more—has been drastically growing in the past decade (Valen- zuela et al., 2012; Rainie et al., 2012; Enikolopov et al., 2019). While online political engagement 1Code and data are publicly available at https://github. com/BunsenFeng/PoliLean. promotes democratic values and diversity of per- spectives, these discussions also reflect and rein- force societal biases—stereotypical generalizations about people or social groups (Devine, 1989; Bargh, 1999; Blair, 2002). Such language constitutes a ma- jor portion of large language models’ (LMs) pre- training data, propagating biases into downstream models. Hundreds of studies have highlighted ethical is- sues in NLP models (Blodgett et al., 2020a; Field et al., 2021; Kumar et al., 2022) and designed syn- thetic datasets (Nangia et al., 2020; Nadeem et al., 2021) or controlled experiments to measure how biases in language are encoded in learned represen- tations (Sun et al., 2019), and how annotator errors in training data are liable to increase unfairness of NLP models (Sap et al., 2019). However, the language of polarizing political issues is particu- larly complex (Demszky et al., 2019), and social biases hidden in language can rarely be reduced to pre-specified stereotypical associations (Joseph and Morgan, 2020). To the best of our knowledge, …

Content truncated to 500 words