NavyTime: Event and Time Ordering from Raw Text

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Seventh International Workshop on Semantic Evaluation (SemEval 2013), pages 73–77, Atlanta, Georgia, June 14-15, 2013. c⃝2013 Association for Computational Linguistics NavyTime: Event and Time Ordering from Raw Text Nathanael Chambers United States Naval Academy Annapolis, MD 21401, USA nchamber@usna.edu Abstract This paper describes a complete event/time ordering system that annotates raw text with events, times, and the ordering relations be- tween them at the SemEval-2013 Task 1. Task 1 is a unique challenge because it starts from raw text, rather than pre-annotated text with known events and times. A working system first identifies events and times, then identifies which events and times should be ordered, and finally labels the ordering relation between them. We present a split classifier approach that breaks the ordering tasks into smaller de- cision points. Experiments show that more specialized classifiers perform better than few joint classifiers. The NavyTime system ranked second both overall and in most subtasks like event extraction and relation labeling. 1 Introduction The SemEval-2013 Task 1 (TempEval-3) contest is the third instantiation of an event ordering challenge. However, it is the first to start from raw text with the challenge to create an end-to-end algorithm for event ordering. Previous challenges included the in- dividual aspects of such a system, including event extraction, timex extraction, and event/time ordering (Verhagen et al., 2007; Verhagen et al., 2010). How- ever, neither task was dependent on the other. This paper presents NavyTime, a system inspired partly by this previous breakup of the tasks. We focus on breaking up the event/time ordering task further, and show that 5 classifiers yield better performance than the traditional 3 (or even 1). The first required steps to annotate a document are to extract its events and time expressions. This pa- per describes a new event extractor with a rich set of contextual features that is a top performer for event attributes at Tempeval-3. We then explore additions to SUTime, a top rule-based extractor for time ex- pressions (Chang and Manning, 2012). However, the core challenge is to link these extracted events and times together. We describe new models for these difficult tasks: (1) identifying ordered pairs, and (2) labeling the ordering relations. Relation identification is rarely addressed in the literature. Given a set of events, which pairs of events are temporally related? Almost all previous work assumes we are given the pairs, and the task is to label the relation (before, after, etc.). Raw text presents a new challenge: extract the relevant pairs before labeling them. We present some of the first results that compare rule-based approaches to trained probabilistic classifiers. These are the first such comparisons to our knowledge. Finally, after relation identification, we label re- lations between the pairs. This is the traditional event ordering task, although we now start from noisy pairs. Our main contribution is to build in- dependent classifiers for intra-sentence event/time pairs. We show improved performance when train- ing these split classifiers. NavyTime’s …

Content truncated to 500 words