GGPONC 2.0 - The German Clinical Guideline Corpus for Oncology: Curation Workflow, Annotation Policy, Baseline NER Taggers

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 3650–3660 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 3650 GGPONC 2.0 — The German Clinical Guideline Corpus for Oncology: Curation Workflow, Annotation Policy, Baseline NER Taggers Florian Borchert1,⋆, Christina Lohr2,♠, Luise Modersohn2,♠, Jonas Witt1, Thomas Langer3, Markus Follmann3, Matthias Gietzelt4,⋆, Bert Arnrich1, Udo Hahn2,♠, Matthieu-P. Schapranow1,⋆ 1Digital Health Center, Hasso Plattner Institute, University of Potsdam, Germany 2Jena University Language & Information Engineering (JULIE) Lab, Friedrich Schiller University Jena, Germany 3German Guideline Program in Oncology, German Cancer Society, Berlin, Germany 4Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School, Hannover, Germany ⋆HIGHMED Consortium of the German Medical Informatics Initiative ♠SMITH Consortium of the German Medical Informatics Initiative {firstname.lastname}@{hpi|uni-jena|mh-hannover}.de, {lastname}@krebsgesellschaft.de Abstract Despite remarkable advances in the development of language resources over the recent years, there is still a shortage of annotated, publicly available corpora covering (German) medical language. With the initial release of the German Guideline Program in Oncology NLP Corpus (GGPONC), we have demonstrated how such corpora can be built upon clinical guidelines, a widely available resource in many natural languages with a reasonable coverage of medical terminology. In this work, we describe a major new release for GGPONC. The corpus has been substantially extended in size and re-annotated with a new annotation scheme based on SNOMED CT top level hierarchies, reaching high inter-annotator agreement (γ = .94). Moreover, we annotated elliptical coordinated noun phrases and their resolutions, a common language phenomenon in (not only German) scientific documents. We also trained BERT-based named entity recognition models on this new data set, which achieve high performance on short, coarse-grained entity spans (F1 = .89), while the rate of boundary errors increases for long entity spans. GGPONC is freely available through a data use agreement. The trained named entity recognition models, as well as the detailed annotation guide, are also made publicly available. Keywords: Clinical Guidelines, Clinical NLP, Annotation, Named Entity Recognition, SNOMED CT, German Medical Language 1. Introduction In the clinical domain, the inability to share language resources and models has been a major obstacle for re- searchers and medical practitioners who want to gain insights from clinical documents through natural lan- guage processing (NLP). This bottleneck is due to pro- tected health information (PHI) in these documents, which allows to identify patients, clinical personnel, or other individuals. Even thorough de-identification efforts targeting PHI items (for a survey, cf. Meystre (2015)) do not guarantee positive votes from ethical boards for allowing strictly anonymized clinical data to pass hospital walls. Likewise, machine learning mod- els trained on these data are usually not shareable, be- cause attackers might re-identify sensitive data from their model parameters (Carlini et al., 2021). Whereas the distribution of real, de-identified clinical documents on a reasonable scale seems out of scope for almost all European countries for the time being, scien- tific research publications are not subject to such distri- bution restrictions. Unfortunately, the vast majority …

Content truncated to 500 words