Sensitive Data Detection and Classification in Spanish Clinical Text: Experiments with BERT

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 4486–4494 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 4486 Sensitive Data Detection and Classification in Spanish Clinical Text: Experiments with BERT Aitor Garc´ıa-Pablos, Naiara Perez, Montse Cuadros SNLT group at Vicomtech Foundation, Basque Research and Technology Alliance (BRTA) Donostia/San-Sebasti´an, 20009, Spain {agarciap, nperez, mcuadros}@vicomtech.org Abstract Massive digital data processing provides a wide range of opportunities and benefits, but at the cost of endangering personal data privacy. Anonymisation consists in removing or replacing sensitive information from data, enabling its exploitation for different purposes while preserving the privacy of individuals. Over the years, a lot of automatic anonymisation systems have been proposed; however, depending on the type of data, the target language or the availability of training documents, the task remains challenging still. The emergence of novel deep-learning models during the last two years has brought large improvements to the state of the art in the field of Natural Language Processing. These advancements have been most noticeably led by BERT, a model proposed by Google in 2018, and the shared language models pre-trained on millions of documents. In this paper, we use a BERT-based sequence labelling model to conduct a series of anonymisation experiments on several clinical datasets in Spanish. We also compare BERT to other algorithms. The experiments show that a simple BERT-based model with general-domain pre-training obtains highly competitive results without any domain specific feature engineering. Keywords: Anonymisation, De-identification, PHI, Clinical Data, BERT 1. Introduction During the first two decades of the 21st century, the sharing and processing of vast amounts of data has become per- vasive. This expansion of data sharing and processing ca- pabilities is both a blessing and a curse. Data helps build better information systems for the digital era and enables further research for advanced data management that ben- efits the society in general. But the use of this very data containing sensitive information conflicts with private data protection, both from an ethical and a legal perspective. There are several application domains on which this situ- ation is particularly acute. This is the case of the medical domain (Abouelmehdi et al., 2018). There are plenty of po- tential applications for advanced medical data management that can only be researched and developed using real data; yet, the use of medical data is severely limited –when not entirely prohibited– due to data privacy protection policies. One way of circumventing this problem is to anonymise the data by removing, replacing or obfuscating the personal information mentioned, as exemplified in Table 1. This task can be done by hand, having people read and anonymise the documents one by one. Despite being a reliable and simple solution, this approach is tedious, expensive, time consuming and difficult to scale to the potentially thousands or millions of documents that need to be anonymised. For this reason, numerous of systems and approaches have been developed during the last decades to attempt to auto- mate the …

Content truncated to 500 words