UFPRSheffield: Contrasting Rule-based and Support Vector Machine Approaches to Time Expression Identification in Clinical TempEval

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 835–839, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics UFPRSheffield: Contrasting Rule-based and Support Vector Machine Approaches to Time Expression Identification in Clinical TempEval Hegler Tissot Federal University of Parana Cel. Franc. H. dos Santos, 100 Curitiba, PR 81531-980, Brazil hctissot@inf.ufpr.br Genevieve Gorrell The University of Sheffield 211 Portobello Sheffield, S1 4DP, UK g.gorrell@shef.ac.uk Angus Roberts The University of Sheffield 211 Portobello Sheffield, S1 4DP, UK angus.roberts@shef.ac.uk Leon Derczynski The University of Sheffield 211 Portobello Sheffield, S1 4DP, UK leon.derczynski@shef.ac.uk Marcos Didonet Del Fabro Federal University of Parana Cel. Franc. H. dos Santos, 100 Curitiba, PR 81531-980, Brazil marcos.ddf@inf.ufpr.br Abstract We present two approaches to time expres- sion identification, as entered in to SemEval- 2015 Task 6, Clinical TempEval. The first is a comprehensive rule-based approach that favoured recall, and which achieved the best recall for time expression identification in Clin- ical TempEval. The second is an SVM-based system built using readily available compo- nents, which was able to achieve a competi- tive F1 in a short development time. We dis- cuss how the two approaches perform relative to each other, and how characteristics of the corpus affect the suitability of different ap- proaches and their outcomes. 1 Introduction SemEval-2015 Task 6, Clinical TempEval (Bethard et al., 2015), was a temporal information extraction task over the clinical domain. The combined University of Sheffield/Federal University of Parana team fo- cused on identification of spans and features for time expressions (TIMEX3), based on specific annotation guidelines (TS and TA subtasks). For time expressions, participants identified ex- pression spans within the text and their corresponding classes: DATE, TIME, DURATION, QUANTIFIER, PREPOSTEXP or SET.1 Participating systems had to annotate timexes according to the guidelines for the annotation of times, events and temporal rela- 1There was no time normalisation task in Clinical TempEval tions in clinical notes – THYME Annotation Guide- lines (Styler et al., 2014) – which is an extension of ISO TimeML (Pustejovsky et al., 2010) developed by the THYME project.2 Further, ISO TimeML ex- tends two other guidelines: a) TimeML Annotation Guidelines (Sauri et al., 2006), and b) TIDES 2005 Standard for the Annotation of Temporal Expressions (Ferro et al., 2005). Clinical TempEval temporal ex- pression results3 were given in terms of Precision, Recall and F1-score for identifying spans and classes of temporal expressions. For Clinical TempEval two datasets were provided. The first was a training dataset comprising 293 doc- uments with a total number 3818 annotated time ex- pressions. The second dataset comprised 150 docu- ments with 2078 timexes. This was used for evalua- tion and was then made available to participants, after evaluations were completed. Annotations identified the span and class of each timex. Table 1 shows the number of annotated timex by class in each dataset. We present a rule-based and a SVM-based ap- proach to time expression identification, and we dis- cuss how they perform relative to each other, and how characteristics of the …

Content truncated to 500 words