PHEE: A Dataset for Pharmacovigilance Event Extraction from Text

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5571 - 5587 December 7-11, 2022 ©2022 Association for Computational Linguistics PHEE: A Dataset for Pharmacovigilance Event Extraction from Text Zhaoyue Sun1, Jiazheng Li1, Gabriele Pergola1, Byron C. Wallace2 Bino John3, Nigel Greene3, Joseph Kim3 and Yulan He1,4,5 1Department of Computer Science, University of Warwick 2Khoury College of Computer Sciences, Northeastern University 3AstraZeneca 4Department of Informatics, King’s College London 5The Alan Turing Institute {Zhaoyue.Sun, Jiazheng.Li, Gabriele.Pergola.1}@warwick.ac.uk {bino.john, nigel.greene, joseph.kim1}@astrazeneca.com b.wallace@northeastern.edu, yulan.he@kcl.ac.uk Abstract The primary goal of drug safety researchers and regulators is to promptly identify adverse drug reactions. Doing so may in turn prevent or reduce the harm to patients and ultimately im- prove public health. Evaluating and monitoring drug safety (i.e., pharmacovigilance) involves analyzing an ever growing collection of sponta- neous reports from health professionals, physi- cians, and pharmacists, and information volun- tarily submitted by patients. In this scenario, facilitating analysis of such reports via automa- tion has the potential to rapidly identify safety signals. Unfortunately, public resources for de- veloping natural language models for this task are scant. We present PHEE, a novel dataset for pharmacovigilance comprising over 5000 annotated events from medical case reports and biomedical literature, making it the largest such public dataset to date. We describe the hierar- chical event schema designed to provide coarse and fine-grained information about patients’ demographics, treatments and (side) effects. Along with the discussion of the dataset, we present a thorough experimental evaluation of current state-of-the-art approaches for biomedi- cal event extraction, point out their limitations, and highlight open challenges to foster future research in this area1. 1 Introduction Pharmacovigilance is the pharmaceutical science that entails monitoring and evaluating the safety and efficiency of medicine use, which is vital for improving public health (World Health Organi- zation, 2004). Unexpected adverse drug effects (ADEs) could lead to considerable morbidity and mortality (Lazarou et al., 1998). It has been re- ported that more than half of ADEs are preventable 1Our data and code is available at https://github.com/ZhaoyueSun/PHEE (Gurwitz et al., 2000). Pharmacovigilance is there- fore important for detecting and understanding ADE-related events, as it may inform clinical prac- tice and ultimately mitigate preventable hazards. Collecting and maintaining the clinical evidence for pharmacovigilance can be difficult because it requires time-consuming manual curation to cap- ture emerging data about drugs (Thompson et al., 2018). Much of this information can be found in unstructured textual data including medical lit- erature, notes in electronic health records (EHR), and social media posts. Using NLP methods to discover and extract adverse drug events from un- structured text may permit efficient monitoring of such sources (Nikfarjam et al., 2015; Huynh et al., 2016; Ju et al., 2020; Wei et al., 2020). Past work has introduced pharmacovigilance cor- pora to support training and evaluation of NLP ap- proaches for ADE extraction. However, most of these datasets (e.g., the ADE corpus; Gurulingappa et al. 2012b) contain annotations only on entities (such as drugs and side effects) and their …

Content truncated to 500 words