Learning to Diagnose: Assimilating Clinical Narratives using Deep Reinforcement Learning

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 895–905, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP Learning to Diagnose: Assimilating Clinical Narratives using Deep Reinforcement Learning Yuan Ling, Sadid A. Hasan, Vivek Datla, Ashequl Qadir, Kathy Lee, Joey Liu, and Oladimeji Farri Artificial Intelligence Lab, Philips Research North America, Cambridge, MA, USA {yuan.ling,sadid.hasan,vivek.datla,ashequl.qadir}@philips.com {kathy.lee 1,joey.liu,dimeji.farri}@philips.com Abstract Clinical diagnosis is a critical and non- trivial aspect of patient care which often requires significant medical research and investigation based on an underlying clin- ical scenario. This paper proposes a novel approach by formulating clinical diagnosis as a reinforcement learning problem. Dur- ing training, the reinforcement learning agent mimics the clinician’s cognitive pro- cess and learns the optimal policy to ob- tain the most appropriate diagnoses for a clinical narrative. This is achieved through an iterative search for candidate diagnoses from external knowledge sources via a sentence-by-sentence analysis of the in- herent clinical context. A deep Q-network architecture is trained to optimize a reward function that measures the accuracy of the candidate diagnoses. Experiments on the TREC CDS datasets demonstrate the effec- tiveness of our system over various non- reinforcement learning-based systems. 1 Introduction Clinical diagnosis is a critical aspect of patient care requiring expert medical knowledge and intu- ition. Given a clinical case narrative such as a pa- tient’s past medical history and current condition, a clinician performs complex cognitive processes to infer the probable diagnosis based on his/her ex- perience or up-to-date knowledge obtained from relevant external resources (Norman et al., 2007). Table 1 shows an example clinical narrative with relevant external knowledge, which suggests that Pulmonary Embolism is the diagnosis for this clin- ical scenario.1 1The clinical narrative with corresponding diagnosis is obtained from the Text REtrieval Conference (TREC) Clin- Clinical Narrative: An 87 yo woman with h/o osteoporosis, DM2, dementia, depression, and anxiety presents s/p fall with evidence of C2 fracture, chest pain, tachycardia, tachypnea, and low blood pressure. External Knowledge (partially shown) From Wikipedia page for Pulmonary Embolism - “Signs and symptoms” Section: Symptoms of pulmonary embolism are typically sudden in onset and may include one or many of the following: dyspnea (shortness of breath), tachypnea (rapid breath- ing), chest pain of a “pleuritic” nature (worsened by breathing), cough and hemoptysis (coughing up blood). From MayoClinic page for Pulmonary Embolism - “Symptoms” Section: Pulmonary embolism symptoms can vary greatly, de- pending on how much of your lung is involved, the size of the clots, and whether you have underlying lung or heart disease. Diagnosis: Pulmonary Embolism Table 1: An example clinical narrative with rele- vant external knowledge and diagnosis. This paper considers the challenge of infer- ring the diagnoses of a patient condition based on available documentation in the Electronic Health Record (EHR), specifically free text clinical re- ports. Earlier work that builds Artificial Intelli- gence (AI) systems to support clinical decision making, mostly uses structured clinical data (e.g. physiological signals, vital signs, lab tests etc.) stored in the EHR …

Content truncated to 500 words