Modelling Temporal Document Sequences for Clinical ICD Coding

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1640–1649 May 2-6, 2023 ©2023 Association for Computational Linguistics Modelling Temporal Document Sequences for Clinical ICD Coding Clarence Boon Liang Ng1 Diogo Santos2 Marek Rei1,2 1 Imperial College London, United Kingdom 2 Transformative AI, United Kingdom {clarence.ng21,marek.rei}@imperial.ac.uk {santos}@transformative.ai Abstract Past studies on the ICD coding problem focus on predicting clinical codes primarily based on the discharge summary. This covers only a small fraction of the notes generated during each hospital stay and leaves potential for im- proving performance by analysing all the avail- able clinical notes. We propose a hierarchical transformer architecture that uses text across the entire sequence of clinical notes in each hospital stay for ICD coding, and incorporates embeddings for text metadata such as their po- sition, time, and type of note. While using all clinical notes increases the quantity of data substantially, superconvergence can be used to reduce training costs. We evaluate the model on the MIMIC-III dataset. Our model exceeds the prior state-of-the-art when using only discharge summaries as input, and achieves further per- formance improvements when all clinical notes are used as input. 1 Introduction ICD (International Classification of Diseases (World Health Organization, 1978)) coding refers to the task where medical professionals classify clinical diagnoses and medical procedures asso- ciated with each patient using standardised tax- onomies, which in turn supports billing, service planning and research. The process is manual and laborious in nature (O’Malley et al., 2005), how- ever there is potential to automate it by identifying relevant information from clinical notes, which are already captured in EHR systems. With this in mind, researchers have started to explore whether machine learning models can succeed at this task (Mullenbach et al., 2018). The current research on the ICD coding task fo- cuses on the extraction of codes from the discharge summary. This document is commonly written at the end of a hospital stay and provides a textual de- scription of the important diagnoses and procedures for a given patient, making it particularly helpful for the task. However, many other clinical notes are also created during the hospital stay, which can pro- vide important details or useful additional context that may be missing from the discharge summary itself. Analysing the full sequence of notes would allow models to make more accurate decisions and make the problem more similar to a real-life setting, where clinicians have to consider all information about a patient for ICD coding, rather than infor- mation only in a single document. In this work we study how the inclusion of clin- ical notes across the entire hospital stay can af- fect performance on the ICD coding task. We pro- pose the Hierarchical Transformers for Document Sequences (HTDS) model, which is an adapta- tion of the hierarchical transformer model (Zhang et al., 2019) for temporal modelling of document sequences. The model takes text and metadata (such as the time and type of note) from a sequence of …

Content truncated to 500 words