Should We Ban English NLP for a Year?

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5254 - 5260 December 7-11, 2022 ©2022 Association for Computational Linguistics Should We Ban English NLP for a Year? Anders Søgaard Dpt. of Computer Science, Pioneer Centre for Artificial Intelligence, and Dpt. of Philosophy University of Copenhagen soegaard@di.ku.dk Abstract Around two thirds of NLP research at top venues is devoted exclusively to developing technology for speakers of English, most speech data comes from young urban speakers, and most texts used to train language models come from male writers. These biases feed into consumer technologies to widen existing in- equality gaps, not only within, but also across, societies. Many have argued that it is almost impossible to mitigate inequality amplification. I argue that, on the contrary, it is quite simple to do so, and that counter-measures would have little-to-no negative impact, except for, perhaps, in the very short term. 1 Inequalities If NLP makes people richer and happier, e.g., by allowing them more free time (Jin et al., 2021), it is unfortunate that NLP predominantly serves the needs of the richest and happiest among us. By and large, NLP supports languages spoken in the world’s wealthiest regions (Blasi et al., 2022). Per- formance disparities within languages and across demographics are also well-documented (Amir et al., 2021; Zhang et al., 2021; Chalkidis et al., 2022) – and performance correlates strongly with income levels (Marrero, 2021; Blasi et al., 2022). Disparities may reflect data imbalances: If training data contains less data from a group, predictions for that group will tend to be worse. But disparities can also result from outlier behavior or higher degrees of variance in groups. Disparities in resources, variation, priority, per- formance, and turn-around go hand-in-hand to cre- ate vicious circles that widen existing equality gaps. Take mobile assistants, for example. Mobile assis- tants help us organize our calendars, place calls, remind us of meetings, etc. Since they are typically speech-operated, their performance depends heav- ily on the performance of available speech recogniz- Figure 1: Validation performance over time when end user group growth is proportional to the performance on this group. Each time step corresponds to the in- clusion of up to 20 new end users. Simulations on four circles datasets generated at random with https: //scikit-learn.org/. The x-axis is time steps, y is classification accuracy. ers. For most languages, speech recognizers were developed for young, urban subpopulations, seen as early adopters. As a consequence, while young urbans reap the benefits of mobile assistants, mo- bile assistants are often much less useful for other demographics (Feng et al., 2021; Markl, 2022). Many have argued that it is almost impossible to mitigate inequality amplification (Fazelpour and Lipton, 2020; Lin and Chen, forthcoming). I ar- gue that it is quite simple to do something, and that doing so would have little-to-no negative im- pact (except for, perhaps, in the very short term). I address two levels of inequality in NLP: inequal- ity across languages and inequality across social …

Content truncated to 500 words