SemEval 2018 Task 6: Parsing Time Normalizations

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 88–96 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics SemEval 2018 Task 6: Parsing Time Normalizations Egoitz Laparra University of Arizona Tucson, AZ 85721, USA laparra@email.arizona.edu Dongfang Xu University of Arizona Tucson, AZ 85721, USA dongfangxu9@email.arizona.edu Steven Bethard University of Arizona Tucson, AZ 85721, USA bethard@email.arizona.edu Ahmed S. Elsayed University of Colorado Boulder Boulder, CO 80309 ahmed.s.elsayed@colorado.edu Martha Palmer University of Colorado Boulder Boulder, CO 80309 martha.palmer@colorado.edu Abstract This paper presents the outcomes of the Pars- ing Time Normalization shared task held within SemEval-2018. The aim of the task is to parse time expressions into the compo- sitional semantic graphs of the Semantically Compositional Annotation of Time Expres- sions (SCATE) schema, which allows the rep- resentation of a wider variety of time expres- sions than previous approaches. Two tracks were included, one to evaluate the parsing of individual components of the produced graphs, in a classic information extraction way, and another one to evaluate the quality of the time intervals resulting from the interpretation of those graphs. Though 40 participants reg- istered for the task, only one team submitted output, achieving 0.55 F1 in Track 1 (parsing) and 0.70 F1 in Track 2 (intervals). 1 Introduction The task of extracting and normalizing time ex- pressions (e.g., finding phrases like two days ago and converting them to a standardized form like 2017-07-17) is a fundamental component of any time-aware language processing system. TempE- val 2010 and 2013 (Verhagen et al., 2010; UzZa- man et al., 2013) included a restricted version of a time normalization task as part of their shared tasks. However, the annotation scheme used in these tasks (TimeML; (ISO, 2012)) has some significant lim- itations: it assumes times can be described as a prefix of YYYY-MM-DDTHH:MM:SS (so it can’t represent, e.g., the past three summers), it is unable to represent times that are are relative to events (e.g., three weeks postoperative), and it fails to reflect the compositional nature of time ex- pressions (e.g., that following represents a similar temporal operation in the following day and the following year). This latter issue especially has discouraged machine learning approaches to time normalization; the most accurate systems for nor- malizing times are still based on sets of complex, manually-constructed rules (Bethard, 2013; Lee et al., 2014; Str¨otgen and Gertz, 2015). The Parsing Time Normalizations shared task is a new approach to time normalization based on the Semantically Compositional Annotation of Time Expressions (SCATE) schema (Bethard and Parker, 2016), in which times are annotated as composi- tional time entities. Such entities are more expres- sive, being able to represent many more time ex- pressions, and are more machine-learnable, as they can naturally be viewed as a semantic parsing task. The top of Figure 1 shows an example. Each an- notation in the example corresponds to a formally defined time entity. For instance, the annotation on top of since corresponds to a BETWEEN entity that identifies an interval starting …

Content truncated to 500 words