USFD2: Annotating Temporal Expresions and TLINKs for TempEval-2

Proceedings of the 5th International Workshop on Semantic Evaluation, ACL 2010, pages 337–340, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics USFD2: Annotating Temporal Expresions and TLINKs for TempEval-2 Leon Derczynski Dept of Computer Science University of Sheffield Regent Court 211 Portobello Sheffield S1 4DP, UK leon@dcs.shef.ac.uk Robert Gaizauskas Dept of Computer Science University of Sheffield Regent Court 211 Portobello Sheffield S1 4DP, UK robertg@dcs.shef.ac.uk Abstract We describe the University of Sheffield system used in the TempEval-2 challenge, USFD2. The challenge requires the au- tomatic identification of temporal entities and relations in text. USFD2 identifies and anchors temporal expressions, and also attempts two of the four temporal relation assignment tasks. A rule-based system picks out and an- chors temporal expressions, and a max- imum entropy classifier assigns temporal link labels, based on features that include descriptions of associated temporal signal words. USFD2 identified temporal expres- sions successfully, and correctly classified their type in 90% of cases. Determin- ing the relation between an event and time expression in the same sentence was per- formed at 63% accuracy, the second high- est score in this part of the challenge. 1 Introduction The TempEval-2 (Pustejovsky and Verhagen, 2009) challenge proposes six tasks. Our system tackles three of these: task A – identifying time ex- pressions, assigning TIMEX3 attribute values, and anchoring them; task C – determining the tempo- ral relation between an event and time in the same sentence; and task E – determining the temporal relation between two main events in consecutive sentences. For our participation in the task, we decided to employ both rule- and ML-classifier- based approaches. Temporal expressions are dealt with by sets of rules and regular expressions, and relation labelling performed by NLTK’s1 maxi- mum entropy classifier with rule-based processing applied during feature generation. The features (described in full in Section 2) included attributes 1See http://www.nltk.org/ . from the TempEval-2 training data annotation, augmented by features that can be directly derived from the annotated texts. There are two main aims of this work: (1) to create a rule-based tempo- ral expression annotator that includes knowledge from work published since GUTime (Mani and Wilson, 2000) and measure its performance, and (2) to measure the performance of a classifier that includes features based on temporal signals. Our entry to the challenge, USFD2, is a succes- sor to USFD (Hepple et al., 2007). In the rest of this paper, we will describe how USFD2 is con- structed (Section 2), and then go on to discuss its overall performance and the impact of some internal parameters on specific TempEval tasks. Regarding classifiers, we found that despite us- ing identical feature sets across relation classifi- cation tasks, performance varied significantly. We also found that USFD2 performance trends with TempEval-2 did not match those seen when clas- sifiers were trained on other data while perform- ing similar tasks. The paper closes with comments about future work. 2 System Description The TempEval-2 training and test sets are parti- tioned into data for entity …

Content truncated to 500 words