Bias Measurement
Related Papers
Detecting Independent Pronoun Bias with Partially-Synthetic Data Generation
AbstractWe report that state-of-the-art parsers consistently failed to identify “hers” and “theirs” as pronouns but identified the masculine equivalent “his”. We find that the same biases exist in recent language …...
Bad Seeds: Evaluating Lexical Methods for Bias Measurement
AbstractA common factor in bias measurement methods is the use of hand-curated seed lexicons, but there remains little guidance for their selection. We gather seeds used in prior work, documenting …...
Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics
AbstractMeasuring bias is key for better understanding and addressing unfairness in NLP/ML models. This is often done via fairness metrics, which quantify the differences in a model’s behaviour across a …...
RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models
AbstractText representation models are prone to exhibit a range of societal biases, reflecting the non-controlled and biased nature of the underlying pretraining data, which consequently leads to severe ethical issues …...