How to leverage the multimodal EHR data for better medical prediction?

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4029–4038 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 4029 How to Leverage Multimodal EHR Data for Better Medical Predictions? Bo Yang Sun Yat-sen University yangb65@mail2.sysu.edu.cn Lijun Wu∗ Microsoft Research Asia lijuwu@microsoft.com Abstract Healthcare is becoming a more and more im- portant research topic recently. With the grow- ing data in the healthcare domain, it offers a great opportunity for deep learning to improve the quality of medical service. However, the complexity of electronic health records (EHR) data is a challenge for the application of deep learning. Specifically, the data produced in the hospital admissions are monitored by the EHR system, which includes structured data like daily body temperature, and unstructured data like free text and laboratory measurements. Al- though there are some preprocessing frame- works proposed for specific EHR data, the clin- ical notes that contain significant clinical value are beyond the realm of their consideration. Besides, whether these different data from var- ious views are all beneficial to the medical tasks and how to best utilize these data re- main unclear. Therefore, in this paper, we first extract the accompanying clinical notes from EHR and propose a method to integrate these data, we also comprehensively study the dif- ferent models and the data leverage methods for better medical task prediction. The results on two medical prediction tasks show that our fused model with different data outperforms the state-of-the-art method that without clini- cal notes, which illustrates the importance of our fusion method and the value of clinical note features. Our code is available at https: //github.com/emnlp-mimic/mimic. 1 Introduction Under the serious struggle of the COVID-19, the healthcare research domain has attracted more and more attention nowadays. With the improvement of information technology, many hospitals have begun to use EHR (Electronic Health Record) sys- tems to monitor all the data produced during the entire hospital admission. The large amount of data generated in this process offers an opportunity for ∗ Corresponding Author. Time-Series Data (Continuous Events) Time-Series Data (Discrete Events) ICU Stay ID Clinical Notes Hours 25130 Time-Invariant Data * ** * * ** ** * * * * 1 2 3 4 5 6 Figure 1: An example of an ICU stay’s record within 6 hours. It contains 3 modalities, including time- invariant data, time-series data, and clinical notes. The time-series data can be further split into discrete events and continuous events. deep learning technology to improve healthcare, such as diagnoses prediction (Choi et al., 2016), medication recommendation (Shang et al., 2019), mortality prediction (Tang et al., 2020), and read- mission prediction (Huang et al., 2019). However, comparing to common academic datasets, such as ImageNet (Deng et al., 2009) and WMT (Macháˇcek and Bojar, 2014), real-world EHR data is longitu- dinal, heterogeneous, and multimodal, which pro- poses big challenges to leverage the information included in it. To further understand the complexity of real- world EHR data, we depict the data in Figure 1. The data …

Content truncated to 500 words