Investigating anatomical bias in clinical machine learning algorithms

Findings of the Association for Computational Linguistics: EACL 2023, pages 1398–1410 May 2-6, 2023 ©2023 Association for Computational Linguistics Investigating anatomical bias in clinical machine learning algorithms Jannik Skyttegaard Pedersen* The Maersk Mc-Kinney Moller Institute University of Southern Denmark jasp@mmmi.sdu.dk Martin Sundahl Laursen* The Maersk Mc-Kinney Moller Institute University of Southern Denmark msla@mmmi.sdu.dk Pernille Just Vinholt Department of Clinical Biochemistry Odense University Hospital Anne Bryde Alnor Department of Clinical Biochemistry Odense University Hospital Thiusius Rajeeth Savarimuthu The Maersk Mc-Kinney Moller Institute University of Southern Denmark Abstract Clinical machine learning algorithms have shown promising results and could potentially be implemented in clinical practice to provide diagnosis support and improve patient treat- ment. Barriers for realisation of the algorithms’ full potential include bias which is systematic and unfair discrimination against certain indi- viduals in favor of others. The objective of this work is to measure anatomical bias in clinical text algorithms. We define anatomical bias as unfair algorithmic out- comes against patients with medical conditions in specific anatomical locations. We measure the degree of anatomical bias across two ma- chine learning models and two Danish clinical text classification tasks, and find that clinical text algorithms are highly prone to anatomical bias. We argue that datasets for creating clini- cal text algorithms should be curated carefully to isolate the effect of anatomical location in order to avoid bias against patient subgroups. 1 Introduction Research in clinical machine learning algo- rithms have shown promising results for automat- ing clinical tasks. The algorithms could potentially be implemented in clinical practice to provide diag- nosis support, improve patient treatment and pro- vide time-savings for medical doctors (Topol, 2019; Matheny et al., 2020). However, despite appealing research results, there are currently limited examples of algorithms being successfully deployed into clinical practice (Kelly et al., 2019). Barriers for realisation of the algorithms’ full potential include bias and generali- *Equal contribution sation issues (Char et al., 2018; Hovy and Prabhu- moye, 2021; Carrell et al., 2017). Algorithmic bias can be defined as systematic and unfair discrimination against certain individu- als or groups of individuals in favor of others (Fried- man and Nissenbaum, 1996). Previous studies have raised serious concerns of algorithms that contain age, gender and racial bias (Sun et al., 2019; David- son et al., 2019) — even for algorithms that have been taken into use (Obermeyer et al., 2019). Al- though machine learning algorithms are trained to be able to generalise to previously unseen data, they tend to overfit to the data they have been trained on. As a consequence of this, bias can unintendedly arise if some subgroups of the target population are not represented in the data used to train the algorithm. Moreover, if the training data itself in- clude biases against some populations, e.g. data reflecting a negative attitude against people with disabilities (Hutchinson et al., 2020), these biases might be encoded and reinforced. If biased algorithms are adopted, healthcare sys- tems risk doing injustice to certain patient groups and harming patient safety (Obermeyer …

Content truncated to 500 words