Building Representative Corpora from Illiterate Communities: A Reviewof Challenges and Mitigation Strategies for Developing Countries

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 2176–2189 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 2176 Building Representative Corpora from Illiterate Communities: A Review of Challenges and Mitigation Strategies for Developing Countries Stephanie Hirmer1, Alycia Leonard1, Josephine Tumwesige2, Costanza Conforti2,3 1Energy and Power Group, University of Oxford 2Rural Senses Ltd. 3Language Technology Lab, University of Cambridge stephanie.hirmer@eng.ox.ac.uk Abstract Most well-established data collection methods currently adopted in NLP depend on the as- sumption of speaker literacy. Consequently, the collected corpora largely fail to repre- sent swathes of the global population, which tend to be some of the most vulnerable and marginalised people in society, and often live in rural developing areas. Such underrepre- sented groups are thus not only ignored when making modeling and system design decisions, but also prevented from benefiting from de- velopment outcomes achieved through data- driven NLP. This paper aims to address the under-representation of illiterate communities in NLP corpora: we identify potential biases and ethical issues that might arise when col- lecting data from rural communities with high illiteracy rates in Low-Income Countries, and propose a set of practical mitigation strategies to help future work. 1 Introduction The exponentially increasing popularity of super- vised Machine Learning (ML) in the past decade has made the availability of data crucial to the development of the Natural Language Processing (NLP) field. As a result, much NLP research has focused on developing rigorous processes for col- lecting large corpora suitable for training ML sys- tems. We observe, however, that many best prac- tices for quality data collection make two implicit assumptions: that speakers have internet access and that they are literate (i.e. able to read and often write text effortlessly1). Such assumptions might be reasonable in the context of most High- Income Countries (HICs) (UNESCO, 2018). How- ever, in Low-Income Countries (LICs), and espe- cially in sub-Saharan Africa (SSA), such assump- tions may not hold, particularly in rural developing 1For example, input from speakers is often taken in writing, in response to a written stimulus which must be read. areas where the bulk of the population lives (Roser and Ortiz-Ospina (2016), Figure 1). As a conse- quence, common data collection techniques – de- signed for use in HICs – fail to capture data from a vast portion of the population when applied to LICs. Such techniques include, for example, crowdsourc- ing (Packham, 2016), scraping social media (Le et al., 2016) or other websites (Roy et al., 2020), collecting articles from local newspapers (Mari- vate et al., 2020), or interviewing experts from in- ternational organizations (Friedman et al., 2017). While these techniques are important to easily build large corpora, they implicitly rely on the above- mentioned assumptions (i.e. internet access and literacy), and might result in demographic misrep- resentation (Hovy and Spruit, 2016). In this pa- per, we make a first step towards addressing how to build representative corpora in LICs from il- literate speakers. We believe that this …

Content truncated to 500 words