Temporal Histories of Epidemic Events (THEE): A Case Study in Temporal Annotation for Public Health

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 2223–2230 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 2223 Temporal Histories of Epidemic Events (THEE): A Case Study in Temporal Annotation for Public Health Jingcheng Niu1, Victoria Ng2, Gerald Penn1, Erin E. Rees2 1Department of Computer Science, University of Toronto 2Public Health Risk Sciences Division, Public Health Agency of Canada {niu, gpenn}@cs.toronto.edu, {victoria.ng, erin.rees}@canada.ca Abstract We present a new temporal annotation standard, THEE-TimeML, and a corpus TheeBank enabling precise temporal information extraction (TIE) for event-based surveillance (EBS) systems in the public health domain. Current EBS must estimate the occurrence time of each event based on coarse document metadata such as document publication time. Because of the complicated language and narration style of news articles, estimated case outbreak times are often inaccurate or even erroneous. Thus, it is necessary to create annotation standards and corpora to facilitate the development of TIE systems in the public health domain to address this problem. We will discuss the adaptations that have proved necessary for this domain as we present THEE-TimeML and TheeBank. Finally, we document the corpus annotation process, and demonstrate the immediate benefit to public health applications brought by the annotations. Keywords: Temporal Information Extraction, Event-based Surveillance System, TimeML, Public Heath 1. Introduction Event-based surveillance systems gather publicly available news articles, blogs and social media posts, and then extract epidemic-related information to detect evidence of emerg- ing health threats due to infectious diseases. Most EBS systems (Brownstein et al., 2008; Mawudeku et al., 2013; Odlum and Yoon, 2018; Jordan et al., 2019) must rely on coarse temporal information, such as the document fetch time or import time, to estimate the occurrence time of the events extracted. This compromise often leads to inaccu- racies or even errors when determining the temporal in- formation of individual events, because of the complicated layering of temporal references used in reportage about epi- demics. Meanwhile, fine-grained information extraction is a well- researched area in the information extraction community, especially for temporal information extraction (TIE), which is the primary focus of this work. Several data annota- tion standards have been developed, including TimeML1 (Pustejovsky et al., 2005), ISO-TimeML (Pustejovsky et al., 2010), which is based upon it, and the clinical, domain- specific THYME standard (Styler et al., 2014b). Based on those annotation standards, SemEval has held multiple shared tasks (Verhagen et al., 2007; Verhagen et al., 2010; UzZaman et al., 2013; Bethard et al., 2015; Bethard et al., 2016). An abundance of TIE systems have been de- veloped for the shared tasks, reaching near-human perfor- mance (Chang and Manning, 2012; Chambers, 2013; Lee et al., 2016; Str¨otgen et al., 2013; Lin et al., 2019). We created an annotation standard (THEE-TimeML) and a corpus (TheeBank) based upon it to facilitate the de- velopment of TIE systems for the public health domain. The standard and the corpus are aimed at three primary use cases but should be comprehensive enough for future 1In …

Content truncated to 500 words