Dataset and Enhanced Model for Eligibility Criteria-to-SQL Semantic Parsing

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 5829–5837 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 5829 Dataset and Enhanced Model for Eligibility Criteria-to-SQL Semantic Parsing Xiaojing Yu1, Tianlong Chen1, Zhengjie Yu2, Huiyu Li3, Yang Yang4, Xiaoqian Jiang5, Anxiao Jiang1 1 Texas A&M University, 2 University of Science and Technology of China, 3 UT Southwestern Medical Center, 4 Walmart Technology, 5 University of Texas Health Science Center at Houston {vicky yu, wiwjp619}@tamu.edu, yzjfree@mail.ustc.edu.cn, huiyu.li@utsouthwestern.edu, yang.yang2@walmart.com, xiaoqian.jiang@uth.tmc.edu, ajiang@cse.tamu.edu Abstract Clinical trials often require that patients meet eligibility criteria (e.g., have specific conditions) to ensure the safety and the effectiveness of studies. However, retrieving eligible patients for a trial from the electronic health record (EHR) database remains a challenging task for clinicians since it requires not only medical knowledge about eligibility criteria, but also an adequate understanding of structured query language (SQL). In this paper, we introduce a new dataset that includes the first-of-its-kind eligibility-criteria corpus and the corresponding queries for criteria-to-sql (Criteria2SQL), a task translating the eligibility criteria to executable SQL queries. Compared to existing datasets, the queries in the dataset here are derived from the eligibility criteria of clinical trials and include Order-sensitive, Counting-based, and Boolean-type cases which are not seen before. In addition to the dataset, we propose a novel neural semantic parser as a strong baseline model. Extensive experiments show that the proposed parser outperforms existing state-of-the-art general-purpose text-to-sql models while highlighting the challenges presented by the new dataset. The uniqueness and the diversity of the dataset leave a lot of research opportunities for future improvement. Keywords: Semantic Parsing, Text-to-SQL, Eligibility Criteria 1. Introduction In a clinical trial, eligibility criteria are guidelines for en- rolling patients for a specific study. These guidelines de- cide whether a patient is eligible to participate in the study. Cohort definition, i.e., retrieving patients from an Elec- tronic Health Records (EHR) database based on the eligi- bility criteria manually is a very time-consuming process for the clinical research. Semantically parsing eligibility criteria and translating them to a structured query language (i.e., criteria-to-sql) automatically can accelerate the co- hort definition process significantly. However, the task is highly challenging and requires substantial domain-specific knowledge. Given an EHR database of patients and the eli- gibility criteria for a specific clinical trial, it aims at gener- ating the corresponding SQL statements and retrieving eli- gible patients from the EHR database effectively. Although there have been many works on generating SQL queries from unstructured texts (i.e., text-to-sql) that achieve nice performance on large-scale datasets such as WikiSQL (Zhong et al., 2017) and Spider (Yu et al., 2018b), it remains difficult to learn a semantic parser directly from medical eligibility criteria for the criteria-to-sql task. First, to the best of our knowledge, there is currently no med- ical eligibility-criteria dataset publicly available for di- rectly training an end-to-end semantic parser. Furthermore, the techniques derived from general natural-language-to- sql (NL2SQL) datasets such as WikiSQL do not transfer well to …

Content truncated to 500 words