Quantifying Bias from Decoding Techniques in Natural Language Generation

Proceedings of the 29th International Conference on Computational Linguistics, pages 1311–1323 October 12–17, 2022. 1311 Quantifying Bias from Decoding Techniques in Natural Language Generation Mayukh Das and Wolf-Tilo Balke Institute for Information Systems, TU Braunschweig Mühlenpfordtstraße 23, 38106 Braunschweig, Germany {mayukh,balke}@ifis.cs.tu-bs.de Abstract Natural language generation (NLG) models can propagate social bias towards a particular de- mography. Though several studies investigated bias from data and models, the NLG task dis- tinctively uses stochastic decoders that can pos- itively or negatively impact the bias-sensitive tokens initially predicted by the model. To address this gap in research, we present an ex- tensive analysis of bias from decoding tech- niques for open-domain language generation considering the entire decoding space. We an- alyze to what extent bias metrics like toxicity and sentiment are impacted by the individual components of decoder algorithms. We also analyze the trade-off between bias scores and human-annotated generation quality through- out the decoder space. Together, these methods reveal the imperative of testing inference time bias and provide evidence on the usefulness of inspecting the entire decoding spectrum. 1 Introduction Natural language generation (NLG) techniques pro- vide the backbone for many downstream artificial intelligence applications, such as chat-bots, vir- tual assistance, machine translation, automatic sto- rytelling, text summarization, and writing assis- tants. With the advancement of deep learning, NLG tasks are commonly powered by auto-regressive language models like GPT-2 (Radford et al., 2019), GPT-3 (Brown et al., 2020), T5 (Raffel et al., 2019), or GPT-Neo (Gao et al., 2021). However, language models (LMs) pretrained on large web text corpora are also known to pass on stereotypical associa- tions learned from real-world training data. Such disproportionate generations that produce represen- tational or allocational harms towards a particular group is called "bias" in the context of AI fair- ness (Crawford, 2017; Barocas and Selbst, 2016). Although a moderate amount of studies has been conducted on quantifying bias for natural language understanding (NLU) (Webster et al., 2018; Lu et al., 2018; Cao and Daumé III, 2020; Dev et al., 2019; Nangia et al., 2020; Nadeem et al., 2021; Zhao et al., 2018, 2020), exploring the same for NLG is a nascent, yet active area of research. Indeed, bias can be introduced at various phases of the model’s development and deploy- ment pipeline, such as data, modeling, decoding, evaluation. Much of the work on analyzing bias in NLG focuses on benchmarking biases pertaining to models or training data (Henderson et al., 2018; Sheng et al., 2019, 2020; Habash et al., 2019; Bor- dia and Bowman, 2019; Cercas Curry et al., 2020; Liu et al., 2020; Yeo and Chen, 2020; Dhamala et al., 2021). Yet, up to now work on examining biases from decoder techniques is relatively scarce. However, NLG models distinctively use search, random sampling, entropy (softmax penalty) that changes the distribution of model predicted tokens at each autoregressive time-step. Redistributing the predicted token and inducing randomness dur- ing inference can positively or negatively impact the bias-sensitive tokens initially predicted by the model. Bias-sensitive tokens are …

Content truncated to 500 words