PromptEHR: Conditional Electronic Healthcare Records Generation with Prompt Learning

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2873 - 2885 December 7-11, 2022 ©2022 Association for Computational Linguistics PromptEHR: Conditional Electronic Healthcare Records Generation with Prompt Learning Zifeng Wang1 and Jimeng Sun1,2 1Department of Computer Science, University of Illinois Urbana-Champaign 2Carle Illinois College of Medicine, University of Illinois Urbana-Champaign {zifengw2,jimeng}@illinois.edu Abstract Accessing longitudinal multimodal Electronic Healthcare Records (EHRs) is challenging due to privacy concerns, which hinders the use of ML for healthcare applications. Synthetic EHRs generation bypasses the need to share sensitive real patient records. However, ex- isting methods generate single-modal EHRs by unconditional generation or by longitudi- nal inference, which falls short of low flexibil- ity and makes unrealistic EHRs. In this work, we propose to formulate EHRs generation as a text-to-text translation task by language models (LMs), which suffices to highly flexible event imputation during generation. We also design prompt learning to control the generation con- ditioned by numerical and categorical demo- graphic features. We evaluate synthetic EHRs quality by two perplexity measures accounting for their longitudinal pattern (longitudinal im- putation perplexity, lpl) and the connections cross modalities (cross-modality imputation perplexity, mpl). Moreover, we utilize two ad- versaries: membership and attribute inference attacks for privacy-preserving evaluation. Ex- periments on MIMIC-III data demonstrate the superiority of our methods on realistic EHRs generation (53.1% decrease of lpl and 45.3% decrease of mpl on average compared to the best baselines) with low privacy risks. 1 1 Introduction The prevalence of electronic patient healthcare records fuel the development of machine learn- ing models for many healthcare applications (Choi et al., 2016b,a; Wang et al., 2021a,b; Wang and Sun, 2022a). However, sharing EHR data usually undergoes strict and expensive de-identification and administration processes thus being difficult. Although there have been attempts on perturb- ing potentially identifiable attributes as the de- identification step (Emam et al., 2015), they were 1Software is available at https://github.com/ RyanWangZf/PromptEHR. s 𝐷𝑋!, 𝐷𝑋", 𝐷𝑋# 𝐷𝑋! 𝐷𝑋!, 𝐷𝑋" 𝐷𝑋# 𝑣! 𝑣" 𝑣# Unconditional generation Homogeneous longitudinal generation Existing works 𝐷𝑋!, 𝐷𝑋", 𝐷𝑋# 𝑀𝑒𝑑!, 𝑀𝑒𝑑" 𝑣! 𝐷𝑋!, 𝐷𝑋" 𝑣" 𝑀𝑒𝑑#, 𝑀𝑒𝑑$ (1) Heterogeneous longitudinal generation 𝑣! 𝐷𝑋!, 𝐷𝑋", → 𝐷𝑋# 𝑣! 𝐷𝑋!, 𝐷𝑋", 𝐷𝑋# 𝑀𝑒𝑑!, 𝑀𝑒𝑑" (4) Cross-modal generation (3) Visit completion 𝑣! 𝐷𝑋!, 𝐷𝑋" (2) Conditional generation 𝑎𝑔𝑒, 𝑔𝑒𝑛𝑑𝑒𝑟 Feature 𝑥 𝑣" 𝐷𝑋# PromptEHR Figure 1: A conceptual demonstration of how PromptEHR works more flexible than all existing works. vt indicates the t-th visit; DX, Med are short for di- agnosis and medication events; Red are the targets to generate. Our method (PromptEHR) is amenable to four new conditional generation ways thus more controllable and flexible. argued not immune to the hack for re-identification (El Emam et al., 2011; Choi et al., 2017). Alter- natively, generating synthetic but realistic EHRs can circumvent data leakage while preserving the patterns of real EHRs for further research and de- velopment (Biswal et al., 2020). Deep generative models like GANs (Goodfellow et al., 2014) and VAEs (Kingma and Welling, 2013) have become popular for unconditional EHRs gen- eration …

Content truncated to 500 words