BBQ: A hand-built bias benchmark for question answering

Findings of the Association for Computational Linguistics: ACL 2022, pages 2086 - 2105 May 22-27, 2022 c⃝2022 Association for Computational Linguistics 1 BBQ: A Hand-Built Bias Benchmark for Question Answering Alicia Parrish,1 Angelica Chen,2 Nikita Nangia,2 Vishakh Padmakumar,2 Jason Phang,2 Jana Thompson,2 Phu Mon Htut,2 Samuel R. Bowman1,2,3 1New York University 2New York University 3New York University Dept. of Linguistics Center for Data Science Dept. of Computer Science Correspondence: {alicia.v.parrish, bowman}@nyu.edu Abstract It is well documented that NLP models learn social biases, but little work has been done on how these biases manifest in model out- puts for applied tasks like question answer- ing (QA). We introduce the Bias Benchmark for QA (BBQ), a dataset of question sets con- structed by the authors that highlight attested social biases against people belonging to pro- tected classes along nine social dimensions rel- evant for U.S. English-speaking contexts. Our task evaluates model responses at two levels: (i) given an under-informative context, we test how strongly responses refect social biases, and (ii) given an adequately informative con- text, we test whether the model’s biases over- ride a correct answer choice. We fnd that mod- els often rely on stereotypes when the context is under-informative, meaning the model’s out- puts consistently reproduce harmful biases in this setting. Though models are more accurate when the context provides an informative an- swer, they still rely on stereotypes and average up to 3.4 percentage points higher accuracy when the correct answer aligns with a social bias than when it conficts, with this difference widening to over 5 points on examples target- ing gender for most models tested. Introduction Large language models (LMs) learn social biases present in the world, and the increased use of these systems across different contexts increases the cases where these biases can lead to harm. LMs have been found to reproduce social biases in down- stream tasks such as language generation (Sheng et al., 2019) and coreference resolution (Rudinger et al., 2018). The use of these models in real-world applications therefore risks harming marginalized individuals and groups. However, little work has been done to understand how these biases manifest in the outputs of question-answering (QA) models. To assess these biases in model outputs, we mea- sure biases against a range of social categories and A Christian and a Muslim were both seen feeing from the scene of the crime right after the bomb exploded. Both men were interviewed by the police shortly after and the Christian man confessed to the crime. Ambiguous context Disambiguated context Unknown Who likely planted the bomb? Who was likely running for cover? Negative question Non-negative question Unknown The Christian The Muslim (A) Contexts (B) Questions (C) Correct answers Figure 1: Examples in BBQ come in sets of 4, created by varying the two contexts options in (A), ambiguous context only or ambiguous + disambiguated context, with the two question types in (B), negative question or non-negative question. We then repeat this with “Chris- tian” and “Muslim” switched. The correct …

Content truncated to 500 words