Bias Identification and Attribution in NLP Models With Regression and Effect Sizes

Bias Identification and Attribution in NLP Models With Regression and Effect Sizes Erenay Dayanik, IMS, University of Stuttgart, Germany erenay.dayanik@ims.uni-stuttgart.de Ngoc Thang Vu, IMS, University of Stuttgart, Germany ngoc-thang.vu@ims.uni-stuttgart.de Sebastian Padó, IMS, University of Stuttgart, Germany sebastian.pado@ims.uni-stuttgart.de Abstract There is a growing awareness that many NLP systems incorporate biases of various types (e.g., regarding gender or race) which can cause significant social harm. At the same time, the techniques often used for the statistical analysis of biases in NLP systems are still relatively basic. Typically, studies test for the presence of a significant difference between two levels of a single bias variable (e.g., gender: male vs. female) without attention to potential confounders, and do not quantify the importance of the bias variable. This article proposes to analyze bias in the output of NLP systems using multivariate regression models. Such models provide a robust and more informative alternative which (a) generalizes to multiple bias variables, (b) can take covariates into account, (c) can be combined with measures of effect size to quantify the size of bias. Jointly, these effects contribute to a statistically more robust identification and attribution of bias that can be used to diagnose system behavior and extract informative examples. We demonstrate the benefits of our method by analyzing a range of current NLP models on two tasks, namely one regression task (emotion intensity prediction) and one classification task (coreference resolution). 1 Introduction Machine learning has been a major driver of innovation in natural language processing since the 1990s, but only the last decade has seen the widespread deployment of NLP methods for use by non-experts: Applications such as neural machine translation (Wu et al., 2016) or voice assistants (Këpuska and Bohouta, 2018) are now rou- tinely available through end users’ mobile phones, and NLP methods are increasingly used in domains outside computer science such as police work (Sun et al., 2021) and recruiting (Singh et al., 2010). Such systems are, from a user perspective, black boxes whose predictions are generally taken at face value. This makes the question pertinent to what ex- tent the machine learning methods underlying these NLP models are fair, or, on the contrary, to what ex- tent they are subject to biases which impact their pre- dictions. More formally, Friedman and Nissenbaum (1996) defined biased computer systems as systems that “systematically and unfairly discriminate against certain individuals or groups of individuals in favor of others”; (see Mehrabi et al. (2021) for a very similar definition). Clearly, such biases have the potential to cause concrete harm for the disadvantaged groups or individuals (Ben- der and Friedman, 2018; Blodgett et al., 2020) and must be observed and controlled as far as possible. A practical aspect of bias analysis, which the above definition leaves open, is whether discrimination is mea- sured “in vitro” (at the level of system performance) or “in vivo” (at the level of real world consequences). In line with the majority of NLP studies on bias, the present study focusses on bias measured “in …

Content truncated to 500 words