UtahBMI at SemEval-2016 Task 12: Extracting Temporal Information from Clinical Text

Proceedings of SemEval-2016, pages 1256–1262, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics UtahBMI at SemEval-2016 Task 12: Extracting Temporal Information from Clinical Text Abdulrahman Khalifa University of Utah abdulrahman.aal@utah.edu Sumithra Velupillai KTH, Stockholm/King’s College, London sumithra@kth.se Stephane Meystre University of Utah stephane.meystre@hsc.utah.edu Abstract The 2016 Clinical TempEval continued the 2015 shared task on temporal information extraction with a new evaluation test set. Our team, UtahBMI, participated in all sub- tasks using machine learning approaches with ClearTK (LIBLINEAR), CRF++ and CRF- suite packages. Our experiments show that CRF-based classifiers yield, in general, higher recall for multi-word spans, while SVM-based classifiers are better at predicting correct at- tributes of TIMEX3. In addition, we show that an ensemble-based approach for TIMEX3 could yield improved results. Our team achieved competitive results in each subtask with an F1 75.4% for TIMEX3, F1 89.2% for EVENT, F1 84.4% for event relations with document time (DocTimeRel), and F1 51.1% for narrative container (CONTAINS) relations. 1 Introduction Extracting temporal information from unstructured clinical narratives is an important step towards the accurate construction of a patient timeline over the course of clinical care (Savova et al., 2009), identify- ing and tracking patterns of care that are crucial for decision making (Augusto, 2005; Wang et al., 2008) and identifying cases or cohorts with temporal crite- ria for medical research (Raghavan et al., 2014). In the medical domain, more emphasis has been placed on utilizing temporal information from structured databases (Combi et al., 2010). However, recent developments in Medical Natural Language Pro- cessing (NLP) research has stimulated work in ex- tracting information from unstructured clinical text (Meystre et al., 2008; Velupillai et al., 2015a) and fa- cilitated future directions to extracting temporal in- formation (Zhou and Hripcsak, 2007). The i2b2 series of NLP challenges focused in 2012 on extracting events (problems, treatments and tests), time expressions (date, duration, time and fre- quency) and temporal relations (before, after, over- lap) from a set of annotated discharge summaries. The best performing systems used supervised ma- chine learning approaches, except for time expres- sion identification and normalization where rule- based followed by hybrid approaches were most successful (Sun et al., 2013b; Sun et al., 2013a). In 2015, the SemEval challenge included a Clin- ical TempEval task (Bethard et al., 2015) with sim- ilar objectives to the 2012 i2b2 challenge. The TimeML event and temporal expressions specifica- tion language (Pustejovsky et al., 2010) was adapted to define events, time expressions and relation an- notations suitable for the clinical domain (Styler et al., 2014). The THYME (Temporal Histories of Your Medical Event) corpus is used in the Clinical TempEval challenge. The annotations in this cor- pus introduce the use of narrative containers concept (Pustejovsky and Stubbs, 2011) to reduce the com- plexity of finding temporal relations between every possible pair, and allow rapid discovery through au- tomatic inferences. Each event and time expression is, when possible, assigned a narrative container that defines their temporal span. Groups of events and times within a …

Content truncated to 500 words