SemEval-2010 Task 13: TempEval-2

Proceedings of the 5th International Workshop on Semantic Evaluation, ACL 2010, pages 57–62, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics SemEval-2010 Task 13: TempEval-2 Marc Verhagen†, Roser Saur´ı ‡, Tommaso Caselli∗ and James Pustejovsky† † Computer Science Department, Brandeis University, Massachusetts, USA ‡Barcelona Media, Barcelona, Spain ∗ ILC-CNR, Pisa, Italy marc@cs.brandeis.edu roser.sauri@barcelonamedia.org tommaso.caselli@ilc.cnr.it jamesp@cs.brandeis.edu Abstract Tempeval-2 comprises evaluation tasks for time expressions, events and temporal re- lations, the latter of which was split up in four sub tasks, motivated by the notion that smaller subtasks would make both data preparation and temporal relation extrac- tion easier. Manually annotated data were provided for six languages: Chinese, En- glish, French, Italian, Korean and Spanish. 1 Introduction The ultimate aim of temporal processing is the au- tomatic identification of all temporal referring ex- pressions, events and temporal relations within a text. However, addressing this aim is beyond the scope of an evaluation challenge and a more mod- est approach is appropriate. The 2007 SemEval task, TempEval-1 (Verhagen et al., 2007; Verhagen et al., 2009), was an initial evaluation exercise based on three limited tempo- ral ordering and anchoring tasks that were consid- ered realistic both from the perspective of assem- bling resources for development and testing and from the perspective of developing systems capa- ble of addressing the tasks.1 TempEval-2 is based on TempEval-1, but is more elaborate in two respects: (i) it is a multilin- gual task, and (ii) it consists of six subtasks rather than three. In the rest of this paper, we first introduce the data that we are dealing with. Which gets us in a position to present the list of task introduced by TempEval-2, including some motivation as to why we feel that it is a good idea to split up temporal relation classification into sub tasks. We proceed by shortly describing the data resources and their creation, followed by the performance of the sys- tems that participated in the tasks. 1The Semeval-2007 task was actually known simply as TempEval, but here we use Tempeval-1 to avoid confusion. 2 TempEval Annotation The TempEval annotation language is a simplified version of TimeML.2 using three TimeML tags: TIMEX3, EVENT and TLINK. TIMEX3 tags the time expressions in the text and is identical to the TIMEX3 tag in TimeML. Times can be expressed syntactically by adverbial or prepositional phrases, as shown in the following example. (1) a. on Thursday b. November 15, 2004 c. Thursday evening d. in the late 80’s e. later this afternoon The two main attributes of the TIMEX3 tag are TYPE and VAL, both shown in the example (2). (2) November 22, 2004 type="DATE" val="2004-11-22" For TempEval-2, we distinguish four temporal types: TIME (at 2:45 p.m.), DATE (January 27, 1920, yesterday), DURATION (two weeks) and SET (every Monday morning). The VAL attribute as- sumes values according to an extension of the ISO 8601 standard, as enhanced by TIMEX2. Each document has one special TIMEX3 tag, the Document Creation Time (DCT), which is in- terpreted …

Content truncated to 500 words