On the Impact of Random Seeds on the Fairness of Clinical Classifiers

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3808–3823 June 6–11, 2021. ©2021 Association for Computational Linguistics 3808 On the Impact of Random Seeds on the Fairness of Clinical Classifiers Silvio Amir and Jan-Willem van de Meent and Byron C. Wallace Northeastern University {s.amir,j.vandemeent,b.wallace}@northeastern.edu Abstract Recent work has shown that fine-tuning large networks is surprisingly sensitive to changes in random seed(s). We explore the implica- tions of this phenomenon for model fairness across demographic groups in clinical predic- tion tasks over electronic health records (EHR) in MIMIC-III ––– the standard dataset in clin- ical NLP research. Apparent subgroup perfor- mance varies substantially for seeds that yield similar overall performance, although there is no evidence of a trade-off between overall and subgroup performance. However, we also find that the small sample sizes inherent to looking at intersections of minority groups and some- what rare conditions limit our ability to accu- rately estimate disparities. Further, we find that jointly optimizing for high overall perfor- mance and low disparities does not yield statis- tically significant improvements. Our results suggest that fairness work using MIMIC-III should carefully account for variations in ap- parent differences that may arise from stochas- ticity and small sample sizes. 1 Introduction Fine-tuning pre-trained transformers (Vaswani et al., 2017) such as BERT (Devlin et al., 2019) has become the dominant paradigm in NLP, owing to their performance across a range of downstream tasks. Clinical NLP — in which we often aim to make predictions on the basis of notes in elec- tronic health records (EHRs) — is no exception (Alsentzer et al., 2019). However, fine-tuning large networks is a stochastic process. Performance can vary considerably as a function of hyperparame- ter choice, and many parameter sets can yield the same validation accuracy (i.e., the model is not identifiable), and more generally the problem is underspecified (D’Amour et al., 2020). Recent work has demonstrated that the choice of random seeds alone can have dramatic impact on model performance in NLP and beyond, even when all men women white black asian hispanic group −0.10 −0.05 0.00 0.05 ∆ AUC Shock Figure 1: Differences (∆s) with respect to overall per- formance as a function of random seeds for demo- graphic subgroups on the Shock phenotype classifica- tion task. Points represent results from pairs of seeds with similar (≤0.01 difference in AUC) validation per- formance to the best seeds. other hyper-parameters are kept fixed (Phang et al., 2018; Dodge et al., 2020; D’Amour et al., 2020). In this work, we explore the intersection of ran- domness and fairness in the context of clinical NLP. Fairness is a particularly acute concern in clini- cal predictive tasks, given the potential of such models to influence treatment decisions. This has motivated work investigating biases in predictive models trained over EHR (Zhang et al., 2020; Chen et al., 2018, 2019, 2020a; Pfohl et al., 2020; Chen et al., 2020b; Tripathi et al., 2020). We investigate the …

Content truncated to 500 words