Deep Structured Neural Network for Event Temporal Relation Extraction

Proceedings of the 23rd Conference on Computational Natural Language Learning, pages 666–676 Hong Kong, China, November 3-4, 2019. c⃝2019 Association for Computational Linguistics 666 Deep Structured Neural Network for Event Temporal Relation Extraction Rujun Han∗, I-Hung Hsu∗, Mu Yang {rujunhan, ihunghsu, yangmu} @isi.edu Aram Galstyan, Ralph Weischedel, Nanyun Peng {galstyan, weisched, npeng} @isi.edu Information Sciences Institute, University of Southern California Abstract We propose a novel deep structured learning framework for event temporal relation extrac- tion. The model consists of 1) a recurrent neural network (RNN) to learn scoring func- tions for pair-wise relations, and 2) a struc- tured support vector machine (SSVM) to make joint predictions. The neural network automat- ically learns representations that account for long-term contexts to provide robust features for the structured model, while the SSVM in- corporates domain knowledge such as tran- sitive closure of temporal relations as con- straints to make better globally consistent de- cisions. By jointly training the two com- ponents, our model combines the benefits of both data-driven learning and knowledge ex- ploitation. Experimental results on three high- quality event temporal relation datasets (TCR, MATRES, and TB-Dense) demonstrate that incorporated with pre-trained contextualized embeddings, the proposed model achieves sig- nificantly better performances than the state- of-the-art methods on all three datasets. We also provide thorough ablation studies to in- vestigate our model. 1 Introduction Event temporal relation extraction aims at building a graph where nodes correspond to events within a given text, and edges reflect temporal relations be- tween the events. Figure 1a illustrates an example of such graph for the text shown above. Different types of edges specify different temporal relations: the event filed is SIMULTANEOUS with claiming, overruled is BEFORE claiming, and overruled is also BEFORE filed. Temporal relation extraction is beneficial for many downstream tasks such as question answering, information retrieval, and nat- ural language generation. An event graph can po- ∗ The authors contribute equally, alphabetical order. filed imposed overruled claiming simultaneous before Both U.S. and British officials filed objections to the court's jurisdiction in 1995, claiming Security Council resolutions imposed on Lybia to force the suspects' extradition overruled a 1971 Convention which gives Libya the right to try the men. (a) Ground-Truth Sub-graph filed claiming overruled (b) Local Model Predictions filed claiming overruled (c) Structured Model Predictions Figure 1: An illustration of a paragraph with its partial temporal graph. (a) shows the Ground-Truth tempo- ral graph, in which case temporal relation SIMULTA- NEOUS and BEFORE are presented. (b) and (c) are the local and structured predictions, respectively, for three of the event pairs in (a). Local predictions are in- compatible with temporal transitivity rule: overruled has to be BEFORE claiming if overruled is BEFORE filed and filed is SIMULTANEOUS with claiming. The structured model achieves coherence by reversing the temporal order between claiming and overruled. tentially be leveraged to help time-series forecast- ing and provide guidances for natural language generation. The CaTeRs dataset (Mostafazadeh et al., 2016) which annotates temporal and causal relations is constructed for this …

Content truncated to 500 words