Can Synthetic Text Help Clinical Named Entity Recognition? A Study of Electronic Health Records in French

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2320–2338 May 2-6, 2023 ©2023 Association for Computational Linguistics Can Synthetic Text Help Clinical Named Entity Recognition? A Study of Electronic Health Records in French Nicolas Hiebel†, Olivier Ferret‡, Karën Fort⋆, Aurélie Névéol† †Université Paris Saclay, CNRS, LISN, France ‡Université Paris-Saclay, CEA, List, F-91120, Palaiseau, France ⋆Sorbonne Université / Université de Lorraine, CNRS, Inria, LORIA, France †firstname.lastname@lisn.upsaclay.fr, ‡olivier.ferret@cea.fr, ⋆karen.fort@loria.fr Abstract In sensitive domains, the sharing of corpora is restricted due to confidentiality, copyrights, or trade secrets. Automatic text generation can help alleviate these issues by producing syn- thetic texts that mimic the linguistic properties of real documents while preserving confiden- tiality. In this study, we assess the usability of synthetic corpus as a substitute training corpus for clinical information extraction. Our goal is to automatically produce a clinical case corpus annotated with clinical entities and to evalu- ate it for a named entity recognition (NER) task. We use two auto-regressive neural mod- els partially or fully trained on generic French texts and fine-tuned on clinical cases to pro- duce a corpus of synthetic clinical cases. We study variants of the generation process: (i) fine-tuning on annotated vs. plain text (in that case, annotations are obtained a posteriori) and (ii) selection of generated texts based on mod- els’ parameters and filtering criteria. We then train NER models with the resulting synthetic text and evaluate them on a gold standard clini- cal corpus. Our experiments suggest that syn- thetic text is useful for clinical NER. 1 Introduction The lack of specific resources might be the most common hurdle encountered when working in Nat- ural Language Processing (NLP), whether it be the lack of task-specific resources, domain-specific re- sources, or both. A major challenge for biomedical NLP in languages other than English is the lack of available clinical corpora that can be shared with the community (Névéol et al., 2018). One way of addressing this problem is to gen- erate new data automatically. The performance of recent language models based on the transformer architecture (Vaswani et al., 2017) on various tasks that involve generating text (translation, summa- rization, etc.) makes it possible to use the trans- former’s power to generate text to address resource sparsity. More specifically for generation, large gener- ative language models such as GPT-2 (Radford et al., 2019) and GPT-3 (Brown et al., 2020) have the ability to generate well-written text containing a wealth of information. The knowledge gained dur- ing the pre-training of those large language mod- els could be helpful in more specialized domains where there is not enough data to train a generative model from scratch. Dealing with medical data brings one particu- lar issue. The generated data should not contain sensitive information from the data it is inspired by while remaining as close as possible to it. This means that the evaluation of this task is different from the usual evaluation of text generation. The evaluation of text generation is …

Content truncated to 500 words