Changing the World by Changing the Data

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 2182–2194 August 1–6, 2021. ©2021 Association for Computational Linguistics 2182 Changing the World by Changing the Data Anna Rogers Center for Social Data Science University of Copenhagen arogers@sodas.ku.dk Abstract NLP community is currently investing a lot more research and resources into development of deep learning models than training data. While we have made a lot of progress, it is now clear that our models learn all kinds of spuri- ous patterns, social biases, and annotation ar- tifacts. Algorithmic solutions have so far had limited success. An alternative that is being actively discussed is more careful design of datasets so as to deliver specific signals. This position paper maps out the arguments for and against data curation, and argues that funda- mentally the point is moot: curation already is and will be happening, and it is changing the world. The question is only how much thought we want to invest into that process. 1 Introduction The key ingredient behind the recent successes in NLP is Transformer-based language models. The paradigm of pre-training followed by fine-tuning on downstream tasks was popularized by BERT (Devlin et al., 2019), and is actively developed (Rogers et al., 2020b). In December 2020 the hu- man performance baselines on SuperGLUE (Wang et al., 2019a) were surpassed twice, making the community wonder if it is possible to formulate benchmarks not solvable in this paradigm. However, the successes are not the full story. It is becoming increasingly clear that much of the re- markable performance is down to benchmarks that do not actually require sophisticated verbal reason- ing skills due to annotation artifacts and spurious patterns correlating with the target labels (Guru- rangan et al., 2018; McCoy et al., 2019; Paullada et al., 2020).The social biases in NLP models are also attracting more attention (Sheng et al., 2019; Davidson et al., 2019; Hutchinson et al., 2020). The “garbage in, garbage out" principle suggests that the situation will not change without a dramatic reappraisal of how NLP data is collected, both for pre-training and task-specific resources. But that seemingly uncontroversial conclusion is at the core of the interdisciplinary tension between NLP under- stood as a deep learning (DL) application area, and the more qualitative approaches of computational linguistics and AI ethics. How deep that tension goes is illustrated by the recent heated (and some- times less than professional1) debate around “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? " by Bender, Gebru et al (2021). This position paper brings together the argu- ments for and against curating data2 from linguistic and ethical perspectives (§2). It makes the case that curation is unavoidable and already happening, and that any data choices that we make, explicitly or implicitly, will affect the real world (§3). Thus the debate is only about how much thought we should put into this process. If we are to at least …

Content truncated to 500 words