Hierarchical Annotation for Building A Suite of Clinical Natural Language Processing Tasks: Progress Note Understanding

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 5484–5493 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 5484 Hierarchical Annotation for Building A Suite of Clinical Natural Language Processing Tasks: Progress Note Understanding Yanjun Gao1, Dmitriy Dligach2, Timothy Miller3, Samuel Tesch4, Ryan Laffin4, Matthew M. Churpek1, Majid Afshar1 1ICU Data Science Lab, School of Medicine and Public Health, University of Wisconsin 2Loyola University Chicago 3Boston Children’s Hospital and Harvard Medical School 4School of Medicine and Public Health, University of Wisconsin 1{ygao, mchurpek, mafshar}medicine.wisc.edu 2ddligach@luc.edu 3Timothy.Miller@childrens.harvard.edu 4{sgtesch, rlaffin}@wisc.edu Abstract Applying methods in natural language processing on electronic health records (EHR) data is a growing field. Existing corpus and annotation focus on modeling textual features and relation prediction. However, there is a paucity of annotated corpus built to model clinical diagnostic thinking, a process involving text understanding, domain knowledge abstraction and reasoning. This work introduces a hierarchical annotation schema with three stages to address clinical text understanding, clinical reasoning, and summarization. We created an annotated corpus based on an extensive collection of publicly available daily progress notes, a type of EHR documentation that is collected in time series in a problem-oriented format. The conventional format for a progress note follows a Subjective, Objective, Assessment and Plan heading (SOAP). We also define a new suite of tasks, Progress Note Understanding, with three tasks utilizing the three annotation stages. The novel suite of tasks was designed to train and evaluate future NLP models for clinical text understanding, clinical knowledge representation, inference, and summarization. Keywords: clinical natural language processing, electronic health record, clinical reasoning, corpus 1. Introduction Patients in the hospital have a multidisciplinary team of physicians, nurses, and support staff who attend to their care. As part of this care, providers input daily progress notes to update the diagnoses and treatment plan, and to document changes in the patient’s health status. The electronic health record (EHR) contains these daily progress notes, and they are one of the most frequent note types that carry the most relevant and viewed documentation of a patient’s care (Brown et al., 2014). While EHRs are intended to provide efficient care, they are still riddled with problems of note bloat (copying and pasting), information overload (automatically embedded data and administrative docu- mentation), and poorly organized notes that overwhelm physicians and lead to burnout and, ultimately, ineffi- cient care (Shoolin et al., 2013). Applying methods in natural language processing to the EHR is a growing field with many potential ap- plications in clinical decision support and augmented care. Corpus and annotation on EHR data are created to model semantic features and relation through lin- guistic cues, including relation extraction (Mowery et al., 2008), named entity recognition (Wang, 2009; Pa- tel et al., 2018; Lybarger et al., 2021), question answer- ing (Pampari et al., 2018; Raghavan et al., 2021), natu- ral language inference (Romanov and Shivade, 2018), etc. However, few corpora have been built to model clinical thinking, especially about clinical diagnostic …

Content truncated to 500 words