Does Gender Matter? Towards Fairness in Dialogue Systems

Proceedings of the 28th International Conference on Computational Linguistics, pages 4403–4416 Barcelona, Spain (Online), December 8-13, 2020 4403 Does Gender Matter? Towards Fairness in Dialogue Systems Haochen Liu1, Jamell Dacon1, Wenqi Fan2, Hui Liu1, Zitao Liu3∗, Jiliang Tang1 1 Michigan State University, East Lansing, MI, USA 2 The Hong Kong Polytechnic University, Hong Kong 3 TAL Education Group, Beijing, China {liuhaoc1,daconjam}@msu.edu, wenqifan03@gmail.com, liuhui7@msu.edu, liuzitao@100tal.com, tangjili@msu.edu Abstract Recently there are increasing concerns about the fairness of Artificial Intelligence (AI) in real- world applications such as computer vision and recommendations. For example, recognition algorithms in computer vision are unfair to black people such as poorly detecting their faces and inappropriately identifying them as “gorillas”. As one crucial application of AI, dialogue systems have been extensively applied in our society. They are usually built with real human conversational data; thus they could inherit some fairness issues which are held in the real world. However, the fairness of dialogue systems has not been well investigated. In this paper, we perform a pioneering study about the fairness issues in dialogue systems. In particular, we construct a benchmark dataset and propose quantitative measures to understand fairness in dialogue models. Our studies demonstrate that popular dialogue models show significant prejudice towards different genders and races. Besides, to mitigate the bias in dialogue systems, we propose two simple but effective debiasing methods. Experiments show that our methods can reduce the bias in dialogue systems significantly. The dataset and the implementation are released to foster fairness research in dialogue systems 1. 1 Introduction AI techniques have brought great conveniences to our lives. However, they have been proven to be unfair in many real-world applications such as computer vision (Howard and Borenstein, 2018), audio processing (Rodger and Pendharkar, 2004), and recommendations (Yao and Huang, 2017). In other words, AI techniques may make decisions that are skewed towards certain groups of people in these applications (Mehrabi et al., 2019). In the field of computer vision, some face recognition algorithms fail to detect faces of black users (Rose, 2010) or inappropriately label black people as “gorillas” (Howard and Borenstein, 2018). In the field of audio processing, it is found that voice-dictation systems recognize a voice from a male more accurately than that from a female (Rodger and Pendharkar, 2004). Moreover, when predicting criminal recidivism, risk assessment tools tend to predict that people of some certain races are more likely to commit a crime (Tolan et al., 2019). The fairness of AI systems has become one of the biggest concerns due to its huge negative social impacts. Dialogue systems are important AI applications. They interact with users through human-like conversa- tions to satisfy their needs. Conversational question answering agents provide users with the information they want to find (Saha et al., 2018). Task-oriented dialogue agents, such as Apple Siri and Microsoft Cortana, assist users to complete specific tasks such as trip planning and restaurant reservations (Jurafsky and Martin, 2009). Non-task-oriented dialogue agents, also known as chatbots, are designed to chit-chat with users in …

Content truncated to 500 words