TIPSem (English and Spanish): Evaluating CRFs and Semantic Roles in TempEval-2

Proceedings of the 5th International Workshop on Semantic Evaluation, ACL 2010, pages 284–291, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics TIPSem (English and Spanish): Evaluating CRFs and Semantic Roles in TempEval-2 Hector Llorens, Estela Saquete, Borja Navarro University of Alicante Alicante, Spain {hllorens,stela,borja}@dlsi.ua.es Abstract This paper presents TIPSem, a system to extract temporal information from natural language texts for English and Spanish. TIPSem, learns CRF models from training data. Although the used features include different language analysis levels, the ap- proach is focused on semantic informa- tion. For Spanish, TIPSem achieved the best F1 score in all the tasks. For English, it obtained the best F1 in tasks B (events) and D (event-dct links); and was among the best systems in the rest. 1 Introduction The automatic treatment of time expressions, events and their relations over natural language text consists of making temporal elements ex- plicit through a system that identifies and anno- tates them following a standard scheme. This in- formation is crucial for other natural language pro- cessing (NLP) areas, such as summarization or question answering. The relevance of temporal in- formation has been reflected in specialized confer- ences (Schilder et al., 2007) and evaluation forums (Verhagen et al., 2007). We present a system to tackle the six different tasks related to multilingual temporal information treatment proposed in TempEval-2. Particularly, in this evaluation exercise, TimeML (Pustejovsky et al., 2003) is adopted as temporal annotation scheme. In this manner, the tasks require partic- ipating systems to automatically annotate differ- ent TimeML elements. Firstly, task A consists of determining the extent of time expressions as defined by the TimeML TIMEX3 tag, as well as the attributes “type” and “value”. Secondly, task B addresses the recognition and classification of events as defined by TimeML EVENT tag. Fi- nally, tasks C to F comprise the categorization of different temporal links (TimeML LINKs). Figure 1 illustrates the TimeML elements in a sentence. Figure 1: TimeML example In the context of TempEval-2, we tackle all tasks for English and Spanish with a data-driven system. This consists of CRF models inferred from lexical, syntactic and semantic information of given training data. Our main approach, TIPSem (Temporal Information Processing based on Semantic in- formation), is focused on semantic roles and semantic networks. Furthermore, we present a secondary approach, TIPSem-B (TIPSem- Baseline), which contrary to the former does not consider semantic information. The main objectives of this paper are (1) evalu- ating the performance of TIPSem comparing it to other participating systems and (2) measuring the contribution of semantic information to different TempEval-2 tasks though the comparison between our systems: TIPSem and TIPSem-B. This paper is structured as follows. Our ap- proach to address the TempEval-2 tasks is moti- vated in Section 2 and described in Section 3. The results obtained in the evaluation are shown and analyzed in Section 4. Finally, conclusions are drawn in Section 5. 2 Approach motivation The next two subsections describe the two main characteristics of our approach, CRFs …

Content truncated to 500 words