SUTime: Evaluation in TempEval-3

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Seventh International Workshop on Semantic Evaluation (SemEval 2013), pages 78–82, Atlanta, Georgia, June 14-15, 2013. c⃝2013 Association for Computational Linguistics SUTIME: Evaluation in TempEval-3 Angel X. Chang Stanford University angelx@cs.stanford.edu Christopher D. Manning Stanford University manning@cs.stanford.edu Abstract We analyze the performance of SUTIME, a temporal tagger for recognizing and normal- izing temporal expressions, on TempEval-3 Task A for English. SUTIME is available as part of the Stanford CoreNLP pipeline and can be used to annotate documents with temporal information. Testing on the TempEval-3 eval- uation corpus showed that this system is com- petitive with state-of-the-art techniques. 1 Introduction The importance of modeling temporal information is increasingly apparent in natural language appli- cations, such as information extraction and ques- tion answering. Extracting temporal information re- quires the ability to recognize temporal expressions, and to convert them from text to a normalized form that is easy to process. Temporal tagging systems are designed to address this problem. In this paper, we evaluate the performance of the SUTIME (Chang and Manning, 2012) rule-based temporal tagging system. We evaluate the performance of SUTIME on ex- tracting temporal information in TempEval-3 (Uz- Zaman et al., 2013), which requires systems to auto- matically annotate documents with temporal infor- mation using TimeML (Pustejovsky et al., 2003). The TempEval-3 training data contains gold human annotated data from TimeBank, AQUAINT, and a new dataset of silver data automatically annotated using a combination of TipSem (Llorens et al., 2010) and TRIOS (UzZaman and Allen, 2010), two of the best performing systems from TempEval-2 (Verha- gen et al., 2010). 2 System Description We use the Stanford CoreNLP1 pipeline with SU- TIME to identify and normalize TIMEX32 ex- pressions. SUTIME is incorporated into Stanford CoreNLP as part of the Named Entity Recognition annotator. For TempEval-3, we use the standard set of rules provided with SUTIME. Since SUTIME can also recognize temporal expressions whose values are not specified by TIMEX3, we ran SUTIME in a TIMEX3 compatible mode.3 2.1 SUTime SUTIME is a rule-based temporal tagger built on regular expression patterns over tokens. Tempo- ral expressions are bounded in their complexity, so many of them can be captured using finite automata. As shown by systems such as FASTUS (Hobbs et al., 1997), a cascade of finite automata can be very effective at extracting information from text. With SUTIME, we follow a similar staged strategy of (i) building up patterns over individual words to find numerical expressions; then (ii) using patterns over words and numerical expressions to find sim- ple temporal expressions; and finally (iii) forming composite patterns over the discovered temporal ex- pressions. SUTIME recognizes Time, Duration, Interval, and Set according to the TIMEX3 specification. In 1nlp.stanford.edu/software/corenlp.shtml 2www.timeml.org 3sutime.restrictToTimex3 = true 78 addition, it recognizes nested time expressions and duration ranges. To achieve this it uses a temporal pattern language defined over tokens (a regular ex- pression language for expressing how tokenized text should be mapped to temporal objects). SUTIME is built …

Content truncated to 500 words