Predicting Clinical Trial Results by Implicit Evidence Integration

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 1461–1477, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 1461 Predicting Clinical Trial Results by Implicit Evidence Integration Qiao Jin1∗, Chuanqi Tan2, Mosha Chen2, Xiaozhong Liu3, Songfang Huang2 1School of Medicine, Tsinghua University 2Alibaba Group 3Indiana University Bloomington jqa14@mails.tsinghua.edu.cn {chuanqi.tcq,chenmosha.cms,songfang.hsf}@alibaba-inc.com liu237@indiana.edu Abstract Clinical trials provide essential guidance for practicing Evidence-Based Medicine, though often accompanying with unendurable costs and risks. To optimize the design of clinical tri- als, we introduce a novel Clinical Trial Result Prediction (CTRP) task. In the CTRP frame- work, a model takes a PICO-formatted clinical trial proposal with its background as input and predicts the result, i.e. how the Intervention group compares with the Comparison group in terms of the measured Outcome in the studied Population. While structured clinical evidence is prohibitively expensive for manual collec- tion, we exploit large-scale unstructured sen- tences from medical literature that implicitly contain PICOs and results as evidence. Specif- ically, we pre-train a model to predict the dis- entangled results from such implicit evidence and fine-tune the model with limited data on the downstream datasets. Experiments on the benchmark Evidence Integration dataset show that the proposed model outperforms the base- lines by large margins, e.g., with a 10.7% rel- ative gain over BioBERT in macro-F1. More- over, the performance improvement is also val- idated on another dataset composed of clinical trials related to COVID-19. 1 Introduction Shall COVID-19 patients be treated with hydrox- ychloroquine? In the era of Evidence-Based Medicine (EBM, Sackett 1997), medical prac- tice should be guided by well-designed and well- conducted clinical research, such as randomized controlled trials. However, conducting clinical tri- als is expensive and time-consuming. Furthermore, inappropriately designed studies can be devastating in a pandemic: a high-profile Remdesivir clinical trial fails to achieve statistically significant con- clusions (Wang et al., 2020b), partially because it ∗ Work done during internship at Alibaba DAMO Academy. does not attain the predetermined sample size when “competing with” other inappropriately designed trials that are unlikely to succeed or not so urgent to test (e.g.: physical exercises and dietary treat- ments). Therefore, it is crucial to carefully design and evaluate clinical trials before conducting them. Proposing new clinical trials requires support from previous evidence in medical literature or practice. For example, the World Health Orga- nization (WHO) has launched a global megatrial, Solidarity (WHO, 2020), to prioritize clinical re- sources by recommending only four most promis- ing therapies1. The rationale for this suggestion comes from the integration of evidence that they might be effective against coronaviruses or other related organisms in laboratory or clinical studies (Peymani et al., 2016; Sheahan et al., 2017; Morra et al., 2018). However, manual integration of evi- dence is far from satisfying, as one study reports that about 86.2% of clinical trials fail (Wong et al., 2019) and even some of the Solidarity therapies do not get expected results (Mehra et al., 2020). To assist clinical trial designing, we introduce a novel task: Clinical …

Content truncated to 500 words