Transfer Knowledge from Natural Language to Electrocardiography: Can We Detect Cardiovascular Disease Through Language Models?

Findings of the Association for Computational Linguistics: EACL 2023, pages 442–453 May 2-6, 2023 ©2023 Association for Computational Linguistics Transfer Knowledge from Natural Language to Electrocardiography: Can We Detect Cardiovascular Disease Through Language Models? Jielin Qiu1∗, William Han1∗, Jiacheng Zhu1, Mengdi Xu1, Michael Rosenberg3, Emerson Liu2, Douglas Weber1, Ding Zhao1 1Carnegie Mellon University, 2Allegheny General Hospital, 3University of Colorado Abstract Recent advancements in Large Language Mod- els (LLMs) have drawn increasing attention since the learned embeddings pretrained on large-scale datasets have shown powerful abil- ity in various downstream applications. How- ever, whether the learned knowledge by LLMs can be transferred to clinical cardiology re- mains unknown. In this work, we aim to bridge this gap by transferring the knowledge of LLMs to clinical Electrocardiography (ECG). To ad- dress this problem, we propose an approach for cardiovascular disease diagnosis and auto- matic ECG diagnosis report generation. We also introduce an additional loss function by Optimal Transport (OT) to align the distribu- tion between ECG and language embeddings. The learned embeddings are evaluated on two downstream tasks: (1) automatic ECG diag- nosis report generation, and (2) zero-shot car- diovascular disease detection. Our approach is able to generate high-quality cardiac diagnosis reports and also achieves competitive zero-shot classification performance even compared with supervised baselines, which proves the feasi- bility of transferring knowledge from LLMs to the cardiac domain. 1 Introduction Heart and cardiovascular diseases are the leading global cause of death, with 80% of cardiovascu- lar disease-related deaths due to heart attacks and strokes. The clinical 12-lead ECG, when correctly interpreted, is the primary tool to detect cardiac ab- normalities and heart-related issues. ECG provides unique information about the structure and electri- cal activity of the heart and systemic conditions through changes in the timing and morphology of the recorded waveforms. Achievements of ECG interpretation, such that critical and timely ECG interpretations of cardiac conditions, will lead to efficient and cost-effective intervention. ∗* marked as equal contribution LLM starts from the Transformer model (Vaswani et al., 2017) and grows quickly with a wide range of applications (Devlin et al., 2019; Liu et al., 2019b; Brown et al., 2020). Recently, LLM has shown great potential for accelerating learning in many other domains since the learned embeddings can provide meaningful representation for downstream tasks. Examples include transfer- ring the knowledge of LLM to, i.e., robotics con- trol (Liang et al., 2022; Ahn et al., 2022), mul- timodal reasoning and interaction (Zeng et al., 2022; Zellers et al., 2021), robotics planning (Shah et al., 2022; Kant et al., 2022; Jain et al., 2022), decision-making (Li et al., 2022; Huang et al., 2022), robotics manipulation (Shridhar et al., 2022; Ren et al., 2022; Cui et al., 2022; Tam et al., 2022; Khandelwal et al., 2022), code generation (Fried et al., 2022), laws (Kaplan et al., 2020), computer vision (Radford et al., 2021), and so on. Some previous works explored LLM and biolog- ical protein (Rives et al., 2021), or health records (Yang et al., 2022). However, the …

Content truncated to 500 words