Brundlefly at SemEval-2016 Task 12: Recurrent Neural Networks vs. Joint Inference for Clinical Temporal Information Extraction

Proceedings of SemEval-2016, pages 1274–1279, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics Brundlefly at SemEval-2016 Task 12: Recurrent Neural Networks vs. Joint Inference for Clinical Temporal Information Extraction Jason Alan Fries Stanford University Mobilize Center jason-fries@stanford.edu Abstract We submitted two systems to the SemEval- 2016 Task 12: Clinical TempEval challenge, participating in Phase 1, where we identified text spans of time and event expressions in clinical notes and Phase 2, where we predicted a relation between an event and its parent doc- ument creation time. For temporal entity extraction, we find that a joint inference-based approach using struc- tured prediction outperforms a vanilla recur- rent neural network that incorporates word embeddings trained on a variety of large clin- ical document sets. For document creation time relations, we find that a combination of date canonicalization and distant supervision rules for predicting relations on both events and time expressions improves classification, though gains are limited, likely due to the small scale of training data. 1 Introduction This work discusses two information extraction sys- tems for identifying temporal information in clinical text, submitted to SemEval-2016 Task 12 : Clinical TempEval (Bethard et al., 2016). We participated in tasks from both phases: (1) identifying text spans of time and event mentions; and (2) predicting relations between clinical events and document creation time. Temporal information extraction is the task of constructing a timeline or ordering of all events in a given document. In the clinical domain, this is a key requirement for medical reasoning systems as well as longitudinal research into the progression of disease. While timestamps and the structured na- ture of the electronic medical record (EMR) directly capture some aspects of time, a large amount of in- formation on the progression of disease is found in the unstructured text component of the EMR where temporal structure is less obvious. We examine a deep-learning approach to se- quence labeling using a vanilla recurrent neural net- work (RNN) with word embeddings, as well as a joint inference, structured prediction approach using Stanford’s knowledge base construction framework DeepDive (Zhang, 2015). Our DeepDive applica- tion outperformed the RNN and scored similarly to 2015’s best-in-class extraction systems, even though it only used a small set of context window and dic- tionary features. Extraction performance, however lagged this year’s best system submission. For doc- ument creation time relations, we again use Deep- Dive. Our system examined a simple temporal dis- tant supervision rule for labeling time expressions and linking them to nearby event mentions via infer- ence rules. Overall system performance was better than this year’s median submission, but again fell short of the best system. 2 Methods and Materials Phase 1 of the challenge required parsing clinical documents to identify TIMEX3 and EVENT tempo- ral entity mentions in text. TIMEX3 entities are expressions of time, ranging from concrete dates to phrases describing intervals like “the last few months.” EVENT entities are broadly defined as any- thing relevant to a patient’s clinical timeline, e.g., diagnoses, …

Content truncated to 500 words