RaFoLa: A Rationale-Annotated Corpus for Detecting Indicators of Forced Labour

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 3610–3625 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 3610 RaFoLa: A Rationale-Annotated Corpus for Detecting Indicators of Forced Labour Erick Mendez Guzman, Viktor Schlegel and Riza Batista-Navarro Department of Computer Science, University of Manchester Manchester, United Kingdom {erick.mendezguzman, viktor.schlegel, riza.batista}@manchester.ac.uk Abstract Forced labour is the most common type of modern slavery, and it is increasingly gaining the attention of the research and social community. Recent studies suggest that artificial intelligence (AI) holds immense potential for augmenting anti-slavery action. However, AI tools need to be developed transparently in cooperation with different stakeholders. Such tools are contingent on the availability and access to domain-specific data, which are scarce due to the near-invisible nature of forced labour. To the best of our knowledge, this paper presents the first openly accessible English corpus annotated for multi-class and multi-label forced labour detection. The corpus consists of 989 news articles retrieved from specialised data sources and annotated according to risk indicators defined by the International Labour Organization (ILO). Each news article was annotated for two aspects: (1) indicators of forced labour as classification labels and (2) snippets of the text that justify labelling decisions. We hope that our data set can help promote research on explainability for multi-class and multi-label text classification. In this work, we explain our process for collecting the data underpinning the proposed corpus, describe our annotation guidelines and present some statistical analysis of its content. Finally, we summarise the results of baseline experiments based on different variants of the Bidirectional Encoder Representation from Transformer (BERT) model. Keywords: Forced Labour, Explainable Natural Language Processing, Text Classification, Human Rationales 1. Introduction Forced labour is the most common type of modern slavery, affecting at least 24.9 million people world- wide (Landman and Silverman, 2019). The trail of forced labour and the data that could reveal its inter- national network is spread across many organisational and geographical boundaries (Pasley, 2018). Recent evidence suggests that AI can facilitate efforts to combat modern slavery. For instance, network analysis and anomaly detection have been utilised to identify patterns in financial flows as well as to detect populations targeted for exploitation (Milivojevic et al., 2020; Bliss et al., 2021). Furthermore, AI models have been used to analyse mobile phone data, specifically mobile money, to predict the incidence of forced labour in Africa (Milivojevic et al., 2020). Considering that most of the data regarding modern slavery come in the form of text records, more recent efforts have focused on using Natural Language Processing (NLP) methods to boost the detection of forced labour (Pasley, 2018). Eliminating modern slavery and forced labour has been particularly challenging due to its near-invisible nature (Landman and Silverman, 2019). Nevertheless, NLP methods can identify individuals and patterns from unstructured data to detect possible red flags that indicate exploitation (Bliss et al., 2021). To accom- plish this, NLP models must be anchored to human rights considerations and developed in …

Content truncated to 500 words