Exploiting Partially Annotated Data in Temporal Relation Extraction

Proceedings of the 7th Joint Conference on Lexical and Computational Semantics (*SEM), pages 148–153 New Orleans, June 5-6, 2018. c⃝2018 Association for Computational Linguistics Exploiting Partially Annotated Data for Temporal Relation Extraction Qiang Ning,1 Zhongzhi Yu,2 Chuchu Fan,1 Dan Roth1,2,3 1Department of Electrical and Computer Engineering, 2Department of Computer Science University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA 3Department of Computer Science, University of Pennsylvania, Philadelphia, PA 19104, USA {qning2,zyu19,cfan10}@illinois.edu, danroth@seas.upenn.edu Abstract Annotating temporal relations (TempRel) be- tween events described in natural language is known to be labor intensive, partly because the total number of TempRels is quadratic in the number of events. As a result, only a small number of documents are typically an- notated, limiting the coverage of various lexi- cal/semantic phenomena. In order to improve existing approaches, one possibility is to make use of the readily available, partially annotated data (P as in partial) that cover more docu- ments. However, missing annotations in P are known to hurt, rather than help, existing sys- tems. This work is a case study in exploring various usages of P for TempRel extraction. Results show that despite missing annotations, P is still a useful supervision signal for this task within a constrained bootstrapping learn- ing framework. The system described in this system is publicly available.1 1 Introduction Understanding the temporal information in natu- ral language text is an important NLP task (Ver- hagen et al., 2007, 2010; UzZaman et al., 2013; Minard et al., 2015; Bethard et al., 2016, 2017). A crucial component is temporal relation (TempRel; e.g., before or after) extraction (Mani et al., 2006; Bethard et al., 2007; Do et al., 2012; Chambers et al., 2014; Mirza and Tonelli, 2016; Ning et al., 2017, 2018a,b). The TempRels in a document or a sentence can be conveniently modeled as a graph, where the nodes are events, and the edges are labeled by TempRels. Given all the events in an instance, TempRel annotation is the process of manually la- beling all the edges – a highly labor intensive task due to two reasons. One is that many edges re- quire extensive reasoning over multiple sentences 1https://cogcomp.org/page/publication_ view/832 and labeling them is time-consuming. Perhaps more importantly, the other reason is that #edges is quadratic in #nodes. If labeling an edge takes 30 seconds (already an optimistic estimation), a typi- cal document with 50 nodes would take more than 10 hours to annotate. Even if existing annotation schemes make a compromise by only annotating edges whose nodes are from a same sentence or adjacent sentences (Cassidy et al., 2014), it still takes more than 2 hours to fully annotate a typ- ical document. Consequently, the only fully an- notated dataset, TB-Dense (Cassidy et al., 2014), contains only 36 documents, which is rather small compared with datasets for other NLP tasks. A small number of documents may indicate that the annotated data provide a limited coverage of various lexical and semantic phenomena, since a document is usually “homogeneous” within it- self. In contrast …

Content truncated to 500 words