Prior-informed Distant Supervision for Temporal Evidence Classification

Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 996–1006, Dublin, Ireland, August 23-29 2014. Prior-informed Distant Supervision for Temporal Evidence Classification Ridho Reinanda University of Amsterdam Amsterdam, The Netherlands r.reinanda@uva.nl Maarten de Rijke University of Amsterdam Amsterdam, The Netherlands derijke@uva.nl Abstract Temporal evidence classification, i.e., finding associations between temporal expressions and re- lations expressed in text, is an important part of temporal relation extraction. To capture the variations found in this setting, we employ a distant supervision approach, modeling the task as multi-class text classification. There are two main challenges with distant supervision: (1) noise generated by incorrect heuristic labeling, and (2) distribution mismatch between the target and distant supervision examples. We are particularly interested in addressing the second problem and propose a sampling approach to handle the distribution mismatch. Our prior-informed distant supervision approach improves over basic distant supervision and outperforms a purely super- vised approach when evaluated on TAC-KBP data, both on classification and end-to-end metrics. 1 Introduction Temporal relation extraction is the problem of extracting the temporal extent of relations between entities. A typical solution to the temporal relation extraction problem has three main components: (1) passage retrieval, (2) temporal evidence classification, and (3) temporal evidence aggregation. A community- based effort to evaluate temporal relation extraction was introduced in 2011 as a TAC Knowledge Base Population task: Temporal Slot Filling, or TSF for short (Ji et al., 2011). An illustration of temporal slot filling is as follows. Having identified a per:spouse relation be- tween two entities (Freeman Dyson, Imme Dyson), a system must establish the temporal boundaries from its supporting sentence. In the case of the sentence “In 1958, he married Imme Dyson”, the goal is to find that the relation lasts from 1958 until the present day. Within the TSF setting, the boundaries are represented as beginning and ending intervals in a tuple (T1, T2, T3, T4) instead of an exact time expression, so as to allow uncertainty in the system output. We investigate temporal relation extraction following this setting. We focus on the temporal evidence classification part. One of the challenges with relation extraction is the limited amount of training data available to capture the variations in a target corpus: temporal relation extraction faces the same challenge. Employing distant supervision (Mintz et al., 2009) is a way to address the challenge. But generating example training data in the temporal setting is not straightforward: we have to find not only the query and related entity, but also the time expression, in a single text segment. Employing distant supervision for temporal evidence classification will introduce noise, in the form of labels and additional contexts (e.g., lexical features). A lot of previous work in distant supervision has been dedicated to reducing noise in distant supervision (Bunescu and Mooney, 2007; Riedel et al., 2010; Wei et al., 2012). We are interested in another phenomenon: the class distributions found in training data generated by a distant supervision approach. These distributions become an issue if …

Content truncated to 500 words