Bootstrapping Text Anonymization Models with Distant Supervision

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 4477–4487 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 4477 Bootstrapping Text Anonymization Models with Distant Supervision Anthi Papadopoulou1, Pierre Lison2, Lilja Øvrelid1, Ildik´o Pil´an2 1Language Technology Group, University of Oslo 2Norwegian Computing Center Oslo, Norway {anthip,liljao}@ifi.uio.no {plison,pilan}@nr.no Abstract We propose a novel method to bootstrap text anonymization models based on distant supervision. Instead of requiring manually labeled training data, the approach relies on a knowledge graph expressing the background information assumed to be publicly available about various individuals. This knowledge graph is employed to automatically annotate text documents including personal data about a subset of those individuals. More precisely, the method determines which text spans ought to be masked in order to guarantee k-anonymity, assuming an adversary with access to both the text documents and the background information expressed in the knowledge graph. The resulting collection of labeled documents is then used as training data to fine-tune a pre-trained language model for text anonymization. We illustrate this approach using a knowledge graph extracted from Wikidata and short biographical texts from Wikipedia. Evaluation results with a RoBERTa-based model and a manually annotated collection of 553 summaries showcase the potential of the approach, but also unveil a number of issues that may arise if the knowledge graph is noisy or incomplete. The results also illustrate that, contrary to most sequence labeling problems, the text anonymization task may admit several alternative solutions. Keywords: text anonymization, distant supervision, data privacy, neural language models 1. Introduction Personal data is ubiquitous in text documents. Due to this presence of personal information, many text sources fall under the scope of data protection regu- lations such as the General Data Protection Regulation (GDPR) recently introduced in Europe (GDPR, 2016). As a consequence, they cannot be shared with third parties (or even used for other purposes than the one originally intended when collecting the data) without a proper legal ground, such as the explicit consent of the individuals. In case obtaining the consent of all those individuals is unfeasible (for instance because there is not practical way of contacting the individuals in question), an alter- native is to anonymize the data to ensure those individ- uals can no longer be identified. Anonymization is of- ten defined as the complete and irreversible process of removing all Personally Identifiable Information (PII) from a dataset (Elliot et al., 2016). Such PII includes both direct identifiers such as person names or pass- port numbers, but also more indirect information such as date of birth, gender or nationality that can also lead to (re-)identification when combined with one another (Domingo-Ferrer et al., 2016). The anonymization of text data is, however, a difficult challenge for which many open questions remain (Li- son et al., 2021). One important problem is the lack of labeled corpora for this task, making it difficult to train data-driven text anonymization models in many domains. Setting up an anonymization task is costly …

Content truncated to 500 words