Medical Crossing: a Cross-lingual Evaluation of Clinical Entity Linking

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 4212–4220 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 4212 Medical Crossing: a Cross-lingual Evaluation of Clinical Entity Linking Anton Alekseev1,2, Zulfat Miftahutdinov3, Elena Tutubalina4,5, Artem Shelmanov6, Vladimir Ivanov7, Vladimir Kokh8, Alexander Nesterov8 Manvel Avetisian8, Andrey Chertok5,6, Sergey Nikolenko1 1Steklov Mathematical Institute at St. Petersburg, St. Petersburg, Russia 2SPbSU, St. Petersburg, Russia 3Kazan Federal University, Kazan, Russia 4HSE University, Moscow, Russia 5Sber AI, Moscow, Russia 7Innopolis University, Innopolis, Russia6AIRI, Moscow, Russia 8Sber AI Lab, Moscow, Russia {anton.m.alexeyev, zulfatmi, tutubalinaev, artemshelmanov, nomemm}@gmail.com {kokh.v.n, ainesterov, avetisian.m.s, achertok}@sberbank.ru sergey@logic.pdmi.ras.ru Abstract Medical data annotation requires highly qualified expertise. Despite the efforts devoted to medical entity linking in different languages, available data is very sparse in terms of both data volume and languages. In this work, we establish benchmarks for cross-lingual medical entity linking using clinical reports, clinical guidelines, and medical research papers. We present a test set filtering procedure designed to analyze the “hard cases” of entity linking approaching zero-shot cross-lingual transfer learning, evaluate state-of-the-art models, and draw several interesting conclusions based on our evaluation results. Keywords: medical entity linking, embeddings, linking evaluation, cross-lingual methods, zero-shot learning 1. Introduction Entity linking is the task of establishing correspon- dences between free-form text mentions and a formal- ized list of concepts (Shen et al., 2014; Sevgili et al., 2020). In this work, we consider medical entity linking – the task where entity mentions are mapped against a large set of medical concept names and their con- cept unique identifiers (CUIs). The biomedical do- main is characterized by extensive dictionaries of con- cepts such as the Unified Medical Language System (UMLS) (Bodenreider, 2004), Medical Subject Head- ings (MeSH) (Coletti and Bleich, 2001), Systematized Nomenclature of Medicine Clinical Terms (SNOMED- CT) (Spackman et al., 1997), or Medical Dictionary for Regulatory Activities (MedDRA) (Brown et al., 1999), and a high variation of mentions. Early models for biomedical entity linking commonly used classification type losses (Rios and Kavuluru, 2018; Miftahutdinov and Tutubalina, 2019; Lou et al., 2020) that work well on narrow benchmarks but often lead to significant performance degradation on other domains and structurally different texts. Modern ap- proaches usually employ similarity between embed- dings (distributed representations) of words and con- cepts. From classical tf-idf and word2vec embed- dings (Aronson, 2001; Ghiasvand and Kate, 2014; Van Mulligen et al., 2016; Leaman and Lu, 2016; Dermouche et al., 2016), entity linking systems have evolved to leverage vector representations constructed by deep neural models that take advantage of self- attention (Vaswani et al., 2017) and a BERT-like rank- ing architecture (Zhu et al., 2019; Sung et al., 2020; Tu- tubalina et al., 2020). We especially note the Biomed- ical Named Encoder (BNE) (Phan et al., 2019) and BioSyn (Sung et al., 2020) – a Transformer model based on BioBERT (Lee et al., 2020). Along with the progress of embedding techniques and neural architectures, the reported performance of state- of-the-art entity linking models has …

Content truncated to 500 words