Cross-Lingual Knowledge Transfer for Clinical Phenotyping

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 900–909 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 900 Cross-Lingual Knowledge Transfer for Clinical Phenotyping Jens-Michalis Papaioannou∗, Paul Grundmann∗, Betty van Aken∗, Athanasios Samaras† Ilias Kyparissidis‡, George Giannakoulas†, Felix Gers∗, Alexander Löser∗ ∗DATEXIS, Berliner Hochschule für Technik (BHT), Germany, †First Department of Cardiology, AHEPA University Hospital, Aristotle University of Thessaloniki, Greece ‡Laboratory of Medical Physics and Digital Innovation, Aristotle University of Thessaloniki, Greece, {michalis.papaioannou, paul.grundmann, bvanaken, gers, aloeser}@bht-berlin.de {asamaraa, ggiannakoulas}@auth.gr {iliaskypkok}@for.auth.gr Abstract Clinical phenotyping enables the automatic extraction of clinical conditions from patient records, which can be beneficial to doctors and clinics worldwide. However, current state-of-the-art models are mostly applicable to clinical notes written in English. We therefore investigate cross-lingual knowledge transfer strategies to execute this task for clinics that do not use the English language and have a small amount of in-domain data available. We evaluate these strategies for a Greek and a Spanish clinic leveraging clinical notes from different clinical domains such as cardiology, oncology and the ICU. Our results reveal two strategies that outperform the state-of-the-art: Translation-based methods in combination with domain-specific encoders and cross-lingual encoders plus adapters. We find that these strategies perform especially well for classifying rare phenotypes and we advise on which method to prefer in which situation. Our results show that using multilingual data overall improves clinical phenotyping models and can compensate for data sparseness. Keywords: Document Classification, Text categorisation, Multilinguality, Neural language representation models 1. Introduction Clinical phenotyping from text. Clinical informa- tion extraction has the potential to support clinicians in their daily work. Phenotyping—the extraction of patient conditions from text—in particular, can help clinicians to summarize patient states and to find simi- lar patients more easily. Recent years have seen many advancements in automatic phenotyping from clinical text (Mulyar et al., 2019; Zhang et al., 2021), most of them based on neural networks. Especially domain- specific pre-training of such networks has shown to be of great benefit in clinical information extrac- tion (Li et al., 2020; Gu et al., 2020). However, current state-of-the-art models are mostly exclusively applica- ble to English text, due to the large amount of both labeled and unlabeled clinical text resources in English. The need for cross-lingual transfer. We want to in- vestigate how to leverage the clinical knowledge en- coded in English and other languages to the benefit of (low-resource) target languages. This scenario is com- monly described as cross-lingual knowledge transfer (Ponti et al., 2020). Medicine is increasingly trans- formed by globalisation (Labonté et al., 2011) and knowledge collected in different languages has signif- icant impact on local decisions (as seen during the Covid-19 pandemic). However, cross-lingual knowl- edge transfer in the clinical domain remains largely un- explored. Our work takes the perspective of clinicians working with non-English clinical notes. In particu- lar we evaluate on medical texts based on Spanish and Greek in which there are far less such resources openly available. Which cross-lingual approach …

Content truncated to 500 words