Annotating Logical Forms for EHR Questions

Annotating Logical Forms for EHR Questions Kirk Roberts†, Dina Demner-Fushman‡ † School of Biomedical Informatics University of Texas Health Science Center at Houston Houston TX, USA kirk.roberts@uth.tmc.edu ‡ Lister Hill National Center for Biomedical Communications National Library of Medicine National Institutes of Health Bethesda MD, USA ddemner@mail.nih.gov Abstract This paper discusses the creation of a semantically annotated corpus of questions about patient data in electronic health records (EHRs). The goal is to provide the training data necessary for semantic parsers to automatically convert EHR questions into a structured query. A layered annotation strategy is used which mirrors a typical natural language processing (NLP) pipeline. First, questions are syntactically analyzed to identify multi-part questions. Second, medical concepts are recognized and normalized to a clinical ontology. Finally, logical forms are created using a lambda calculus representation. We use a corpus of 446 questions asking for patient-specific information. From these, 468 specific questions are found containing 259 unique medical concepts and requiring 53 unique predicates to represent the logical forms. We further present detailed characteristics of the corpus, including inter-annotator agreement results, and describe the challenges automatic NLP systems will face on this task. Keywords: question answering, semantic parsing, electronic health records 1. Introduction Over the past few years, the adoption of electronic health records (EHRs) has grown remarkably in the United States (Office of the National Coordinator for Health Information Technology, 2014) and many other developed countries as well (Neumann, 2010). However, many usability issues with EHRs are a barrier to their effective use (Zhang and Walji, 2014), including difficulty accessing the information stored in EHRs. Natural language question answering (QA) provides an intuitive interface for retrieving EHR data by reducing the need to understand the internal organization of the data. However, since this data is stored in both un- structured text and structured databases, a deep semantic understanding of EHR questions is necessary for an effec- tive QA system. Consider the following questions: (1) What is her blood pressure and how low has it been? (2) Is he on any inotropes? (3) Is she wheezing this morning? (4) Who is the patient’s nephrologist? For a complete semantic understanding of these questions, several types of linguistic processing must be performed. First, Question (1) must be decomposed into two distinct questions. Second, medical concepts (“blood pressure”, “inotropes”, “wheezing”, and “nephrologist”) must be rec- ognized and normalized to an ontology consistent with how the data is stored in EHRs. Third, the phrase “this morning” must be recognized as an explicit temporal constraint on the answer space. Finally, the questions must be converted to a logical form that captures their full meaning. This includes many other linguistic tasks, such as the implicit temporal semantics in each question. For the questions above, that meaning could be represented by the following lambda cal- culus forms: (1a’) latest(λx.has test(x, 0005823, visit)) (1b’) min(λx.has test(x, 0005823, visit)) (2’) δ(λx.has treatment(x, 0304509, status)) (3’) δ(λx.has problem(x, 0043144, status) ∧ time within(x, ‘this morning’)) (4’) latest(λx.has doctor(x, 0260039, history)) These logical …

Content truncated to 500 words