Re-contextualizing Fairness in NLP: The Case of India

Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 727–740 November 20–23, 2022. ©2022 Association for Computational Linguistics 727 Re-contextualizing Fairness in NLP: The Case of India Shaily Bhatt Google Research shailybhatt@google.com Sunipa Dev Google Research sunipadev@google.com Partha Talukdar Google Research partha@google.com Shachi Dave* Google Research shachi@google.com Vinodkumar Prabhakaran* Google Research vinodkpg@google.com Abstract Recent research has revealed undesirable bi- ases in NLP data and models. However, these efforts focus of social disparities in West, and are not directly portable to other geo-cultural contexts. In this paper, we focus on NLP fair- ness in the context of India. We start with a brief account of the prominent axes of so- cial disparities in India. We build resources for fairness evaluation in the Indian context and use them to demonstrate prediction bi- ases along some of the axes. We then delve deeper into social stereotypes for Region and Religion, demonstrating its prevalence in cor- pora and models. Finally, we outline a holis- tic research agenda to re-contextualize NLP fairness research for the Indian context, ac- counting for Indian societal context, bridging technological gaps in NLP capabilities and re- sources, and adapting to Indian cultural values. While we focus on India, this framework can be generalized to other geo-cultural contexts. 1 Introduction While Natural Language Processing (NLP) has seen impressive advancements recently (Devlin et al., 2018a; Raffel et al., 2019; Brown et al., 2020; Chowdhery et al., 2022), it has also been demon- strated that language technologies may capture, propagate, and amplify societal biases (Blodgett et al., 2020). Although NLP is adopted globally, most studies on assessing and mitigating biases are in the Western context,1 focusing on axes of disparities in the West, relying on Western data and justice norms, and are not directly portable to non-Western contexts (Sambasivan et al., 2021). This is especially troubling for India, a pluralis- tic nation of 1.4 billion people, with fast-growing investments in NLP from the government and the 1We use Western or the West to refer to the regions, nations & states consisting of Europe, the U.S., Canada, and Australa- sia, and their shared norms, values, customs, religious beliefs, & political systems (Kurth, 2003). private sector.2 There is commendable recent work on fairness in NLP models for Indian languages such as Hindi, Bengali, and Telugu (Pujari et al., 2019; Malik et al., 2021; Gupta et al., 2021). But, for a nation with many religions, ethnicities, and cultures, re-contextualizing NLP fairness needs to account for the various axes of social disparities in the Indian society, their proxies in language data, the disparate NLP capabilities in Indian languages, and the (lack of) resources for bias evaluation. Sambasivan et al. (2021) proposed a research agenda for AI fairness for India based on interviews of 36 experts on Indian society and technology. In this paper, we build on their work with a focus on NLP. We start with …

Content truncated to 500 words