Dense Event Ordering with a Multi-Pass Architecture

Dense Event Ordering with a Multi-Pass Architecture Nathanael Chambers United States Naval Academy nchamber@usna.edu Taylor Cassidy Army Research Lab IBM Research taylorcassidy64@gmail.com Bill McDowell Carnegie Mellon University forkunited@gmail.com Steven Bethard University of Alabama at Birmingham bethard@cis.uab.edu Abstract The past 10 years of event ordering research has focused on learning partial orderings over document events and time expressions. The most popular corpus, the TimeBank, contains a small subset of the possible ordering graph. Many evaluations follow suit by only testing certain pairs of events (e.g., only main verbs of neighboring sentences). This has led most research to focus on specific learners for par- tial labelings. This paper attempts to nudge the discussion from identifying some relations to all relations. We present new experiments on strongly connected event graphs that con- tain ∼10 times more relations per document than the TimeBank. We also describe a shift away from the single learner to a sieve-based ar- chitecture that naturally blends multiple learn- ers into a precision-ranked cascade of sieves. Each sieve adds labels to the event graph one at a time, and earlier sieves inform later ones through transitive closure. This paper thus de- scribes innovations in both approach and task. We experiment on the densest event graphs to date and show a 14% gain over state-of-the-art. 1 Introduction Event ordering in the NLP community usually refers to a specific labeling task: given pairs of events and time expressions, choose ordering relations for them. The TimeBank corpus (Pustejovsky et al., 2003) pro- vided a corpus of labeled pairs that motivated ma- chine learning approaches to this task. However, only a small subset of possible pairs are labeled. The annotators identified the most central and obvious re- lations, and left the rest unlabeled. Partly because of this, there has been little research into complete event graphs. This paper describes our recent exploration into the dramatic changes required of both learning algorithms and experimental setup when faced with the task of a more complete graph labeling. The most obvious shift required of a complete graph labeling algorithm is that of relation identifica- tion. Research on the TimeBank and in the TempEval contests has largely focused on relation classification. The event pairs are given, and the task is to classify them. We are now forced to first determine which events should be paired up before classification. Our experiments here fully integrate and evaluate both identification and classification. We are not the first to approach relation identifi- cation with classification, but this paper is the first to directly and comprehensively address it. Most recently, TempEval 3 (UzZaman et al., 2013a) pro- posed a labeling of raw text without prior relation identification, but the challenge ultimately relied on the TimeBank. Systems were only evaluated on its subset of labeled event pairs. This meant that relation identification was largely ignored. The top system op- timized only relation classification and intentionally left many pairs unlabeled (Bethard, 2013). This paper presents CAEVO, a CAscading EVent Ordering architecture. It is a novel …

Content truncated to 500 words