JU_CSE_TEMP: A First Step towards Evaluating Events, Time Expressions and Temporal Relations

Proceedings of the 5th International Workshop on Semantic Evaluation, ACL 2010, pages 345–350, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics JU_CSE_TEMP: A First Step towards Evaluating Events, Time Ex- pressions and Temporal Relations Anup Kumar Kolya1, Asif Ekbal2 and Sivaji Bandyopadhyay3 1,3Department of Computer Science and Engineering, Jadavpur University, Kolkata-700032, India 2Department of Computational Linguistics, Heidelberg University, Heidelberg-69120, Germany Email: anup.kolya@gmail.com1, asif.ekbal@gmail.com2 and sivaji_cse_ju@yahoo.com3 Abstract Temporal information extraction is a popular and interesting research field in the area of Natural Language Processing (NLP). In this paper, we report our works on TempEval-2 shared task. This is our first participation and we participated in all the tasks, i.e., A, B, C, D, E and F. We develop rule-based systems for Tasks A and B, whereas the remaining tasks are based on a machine learning approach, namely Conditional Random Field (CRF). All our systems are still in their development stages, and we report the very initial results. Evaluation results on the shared task English datasets yield the precision, recall and F-measure values of 55%, 17% and 26%, respectively for Task A and 48%, 56% and 52%, respec- tively for Task B (event recognition). The rest of tasks, namely C, D, E and F were evaluated with a relatively simpler metric: the number of correct answers di- vided by the number of answers. Experi- ments on the English datasets yield the accuracies of 63%, 80%, 56% and 56% for tasks C, D, E and F, respectively. 1 Introduction Temporal information extraction is, nowadays, a popular and interesting research area of Natural Language Processing (NLP). Generally, events are described in different newspaper texts, sto- ries and other important documents where events happen in time and the temporal location and ordering of these events are specified. One of the important tasks of text analysis clearly re- quires identifying events described in a text and locating these in time. This is also important in a wide range of NLP applications that include temporal question answering, machine transla- tion and document summarization. In the literature, temporal relation identifica- tion based on machine learning approaches can be found in Boguraev et el. (2005), Mani et al. (2006), Chambers et al. (2007) and some of the TempEval 2007 participants (Verhagen et al., 2007). Most of these works tried to improve classification accuracies through feature engi- neering. The performance of any machine learn- ing based system is often limited by the amount of available training data. Mani et al. (2006) in- troduced a temporal reasoning component that greatly expands the available training data. The training set was increased by a factor of 10 by computing the closure of the various temporal relations that exist in the training data. They re- ported significant improvement of the classifica- tion accuracies on event-event and event-time relations. Their experimental result showed the accuracies of 62.5%-94.95% and 73.68%- 90.16% for event-event and event-time relations, respectively. However, this has two shortcom- ings, namely feature vector duplication caused by the data normalization process and …

Content truncated to 500 words