At the Intersection of NLP and Sustainable Development: Exploring the Impact of Demographic-Aware Text Representations in Modeling Value on a Corpus of Interviews

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 2007–2021 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 2007 At the Intersection of NLP and Sustainable Development: Exploring the Impact of Demographic-Aware Text Representations in Modeling Value on a Corpus of Interviews. Goya van Boven1, Stephanie Hirmer2,4, Costanza Conforti3 1 Department of Information and Computing sciences, Utrecht University 2 Energy and Power Group, University of Oxford 3 Language Technology Lab, University of Cambridge 4 Rural Senses Ltd. j.g.vanboven@students.uu.nl Abstract This research explores automated text classification using data from Low– and Middle–Income Countries (LMICs). In particular, we explore enhancing text representations with demographic information of speakers in a privacy-preserving manner. We introduce the Demographic-Rich Qualitative UPV-Interviews Dataset (DR-QI), a rich dataset of qualitative interviews from rural communities in India and Uganda. The interviews were conducted following the latest standards for respectful interactions with illiterate speakers (Hirmer et al., 2021a). The interviews were later sentence-annotated for Automated User-Perceived Value (UPV) Classification (Conforti et al., 2020), a schema that classifies values expressed by speakers, resulting in a dataset of 5,333 sentences. We perform the UPV classification task, which consists of predicting which values are expressed in a given sentence, on the new DR-QI dataset. We implement a classification model using DistilBERT (Sanh et al., 2019), which we extend with demographic information. In order to preserve the privacy of speakers, we investigate encoding demographic information using autoencoders. We find that adding demographic information improves performance, even if such information is encoded. In addition, we find that the performance per UPV is linked to the number of occurrences of that value in our data. Keywords: Demographic-Aware Text Representation, Multi-Class Multi-Label Text Classification, Privacy-Preserving NLP, Computational Social Science and Cultural Analytics 1. Introduction The interplay between language use and demographic factors has been widely studied in sociolinguistics (Labov, 1972; Extra and Gorter, 2001), and has more recently found application in NLP (Hovy, 2015; Vi- jayaraghavan et al., 2017). While this is an impor- tant step in NLP, most research on demographic-aware text representation examines only a handful of fea- tures. These are often modelled separately: how- ever, in reality, identities are composite and stratified, and they result from the complex interplay and mu- tual influence of different demographic elements (Mc- Call, 2005). Studying each element in isolation is therefore too simplistic and ultimately artificial (Her- belot et al., 2012). Moreover, almost all research in this area has focused on High-Income Countries, with a particular strong focus on the US (Blodgett et al., 2016). However, culturally-specific elements play an important role in defining someone’s identity and certain demographic aspects can be even more de- terminant in shaping a person’s language use, partic- ularly in traditional societies where social roles can be more rigid (Jayachandran, 2020). This paper ad- dresses some of the gaps mentioned above and makes the following contributions: (1) We focus on text clas- sification with a rich set of demographic features, in- cluding less often …

Content truncated to 500 words