Delivering Fairness in Human Resources AI: Mutual Information to the Rescue

Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 867–882 November 20–23, 2022. ©2022 Association for Computational Linguistics 867 Delivering Fairness in Human Resources AI: Mutual Information to the Rescue Léo Hemamou iCIMS, 15 rue de Bucarest, 75008 Paris, France. l.hemamou@gmail.com William Coleman iCIMS, Dogpatch Labs, CHQ Building, Custom House Quay, Dublin 1, D01 Y6H7, Ireland. william.coleman@icims.com Abstract Automatic language processing is used fre- quently in the Human Resources (HR) sector for automated candidate sourcing and evalua- tion of resumes. These models often use pre- trained language models where it is difficult to know if possible biases exist. Recently, Mutual Information (MI) methods have demonstrated notable performance in obtaining representa- tions agnostic to sensitive variables such as gender or ethnicity. However, accessing these variables can sometimes be challenging, and their use is prohibited in some jurisdictions. These factors can make detecting and mitigat- ing biases challenging. In this context, we pro- pose to minimize the MI between a candidate’s name and a latent representation of their CV or short biography. This method may mitigate bias from sensitive variables without requiring the collection of these variables. We evaluate this methodology by first projecting the name representation into a smaller space to prevent potential MI minimization problems in high dimensions. 1 Introduction There are numerous examples of Artificial Intel- ligence (AI) systems which fail to mitigate bias contained within datasets used to train models (Mehrabi et al., 2021; Crawford, 2021; Peña et al., 2020; Buolamwini and Gebru, 2018; Holstein et al., 2019). Bias can be introduced via human labelling or via data extracted from existing human pro- cesses which replicates societal biases (Barocas and Selbst, 2016). Left unchecked, machine learn- ing models will reflect directly the data used to train them or possibly even exacerbate the effect of biased data. This is of particular concern in high-risk domains such as Human Resources (HR), where models can be used to assess candidates based on data provided in a Curriculum Vitae (CV) (Sánchez-Monedero et al., 2020a). Large pre-trained language models (LLMs) have been the source of impressive performance gains in recent times on tasks such as question answering (Yan et al., 2021), common-sense reasoning (Wei et al., 2022), computer coding (Xu et al., 2022) and other domains. However, their capabilities are characterized poorly, requiring a greater under- standing of their function to ameliorate potential harms (Srivastava et al., 2022). Fine-tuning LLMs on downstream tasks has become the gold standard for approaching many natural language process- ing (NLP) tasks (Ruder, 2021). However, the na- ture of this workflow means that practitioners who fine-tune such models on downstream tasks have little visibility of the data used to train the origi- nal model purely because of the volume of data involved. This lack of visibility can be problematic given that these models are trained on huge vol- umes of text data which may contain hidden biases …

Content truncated to 500 words