Clinical Text Classification to SNOMED CT Codes Using Transformers Trained on Linked Open Medical Ontologies

Proceedings of Recent Advances in Natural Language Processing, pages 519–526 Varna, Sep 4–6, 2023 https://doi.org/10.26615/978-954-452-092-2_057 519 Clinical Text Classification to SNOMED CT Codes using Transformers Trained on Linked Open Medical Ontologies Anton Hristov1, Petar Ivanov1, Anna Aksenova1, Tsvetan Asamov1, Pavlin Gyurov1, Todor Primov1, Svetla Boytcheva1, 1OntotextAD, Bulgaria petar.ivanov@ontotext.com, anna.aksenova@ontotext.com, tsvetan.asamov@ontotext.com, pavlin.gyurov@ontotext.com, todor.primov@ontotext.com, svetla.boytcheva@ontotext.com Abstract We present an approach for medical text coding with SNOMED CT. Our approach uses publicly available linked open data from terminologies and ontologies as training data for the algo- rithms. We claim that even small training cor- pora made of short text snippets can be used to train models for the given task. We pro- pose a method based on transformers enhanced with clustering and filtering of the candidates. Further, we adopt a classical machine learning approach - support vector classification (SVC) using the transformer embeddings. The result- ing approach proves to be more accurate than the predictions given by Large Language Mod- els. We evaluate on a dataset generated from linked open data for SNOMED codes related to morphology and topography for four use cases. Our transformers-based approach achieves an F1-score of 0.82 for morphology and 0.99 for topography codes. Further, we validate the ap- plicability of our approach in a clinical context using labelled real clinical data that are not used for model training. 1 Introduction Despite being widely applicable in healthcare, med- ical insurance and medical research, medical cod- ing remains an under-automated process. This is mainly due to the huge amount of codes in medical ontologies on one hand and the very limited access to medical texts for training natural language pro- cessing systems on the other. We are presenting research on the clinical text classification task using SNOMED CT1 codes as target values. Although the recent advances in Artificial intelligence (AI) show significant improvement in transformer-based models’ performance on various Natural Language Processing (NLP) tasks, medical coding remains challenging due to the large number of classes in 1https://www.snomed.org/ SNOMED (about 350K). Moreover, such systems need to be precise and reliable, hence they are usu- ally integrated in Hospital information systems or used in Health insurance companies. Thus we pro- pose ML-based approach that is developed on pub- licly available data. In addition, we compare our system to domain-specific Large Language Models (LLMs). 2 Related Work As manual annotation in the biomedical domain is insufficient, there’s a rise in the adoption of ML ap- proaches that leverage clinical text data for task au- tomation, predictive modelling, and knowledge dis- covery (Khattak et al., 2019; Mujtaba et al., 2019). However, as free-text clinical notes are unstruc- tured, and contain spelling errors, abbreviations, and domain-specific terminology (Leaman et al., 2015), the problem of correct information extrac- tion from clinical free-text remains a bottleneck to be properly addressed. The limited scope of available data leads to a lim- ited range of models that can be employed and, con- sequently, to poorer results. This problem can be partially alleviated by using an English-centric mul- tilingual approach …

Content truncated to 500 words