Embedding Time Expressions for Deep Temporal Ordering Models

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4400–4406 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 4400 Embedding time expressions for deep temporal ordering models Tanya Goyal and Greg Durrett Department of Computer Science The University of Texas at Austin tanyagoyal@utexas.edu, gdurrett@cs.utexas.edu Abstract Data-driven models have demonstrated state- of-the-art performance in inferring the tem- poral ordering of events in text. However, these models often overlook explicit tempo- ral signals, such as dates and time windows. Rule-based methods can be used to identify the temporal links between these time expressions (timexes), but they fail to capture timexes’ in- teractions with events and are hard to integrate with the distributed representations of neural net models. In this paper, we introduce a framework to infuse temporal awareness into such models by learning a pre-trained model to embed timexes. We generate synthetic data consisting of pairs of timexes, then train a character LSTM to learn embeddings and clas- sify the timexes’ temporal relation. We evalu- ate the utility of these embeddings in the con- text of a strong neural model for event tempo- ral ordering, and show a small increase in per- formance on the MATRES dataset and more substantial gains on an automatically collected dataset with more frequent event-timex inter- actions.1 1 Introduction Understanding the temporal ordering of events in a document is an important component of document understanding and plays an integral role in tasks such as timeline creation (Do et al., 2012), tempo- ral question answering (Llorens et al., 2015) and causality inference (Mostafazadeh et al., 2016; Ning et al., 2018a). Inferring temporal event or- der is challenging as it often disagrees with the narrative order in text. Past work on temporal re- lation extraction has exploited cues such as global constraints on the temporal graph structure (Bram- sen et al., 2006; Chambers and Jurafsky, 2008; Ning et al., 2017), world knowledge (Ning et al., 1Data and code are available at https://github. com/tagoyal/Temporal-event-ordering 2018b), grouping of events (Tourille et al., 2017), or fusing these cues more effectively with deep models (Meng et al., 2017; Cheng and Miyao, 2017). One key component of temporal under- standing is time expressions (timexes) that help anchor events to the time axis, but few recent sys- tems effectively use the knowledge derivable from time expressions in their models. They either give timexes no special treatment (Ning et al., 2017) or rely on rule-based post-processing modules to remove inconsistencies with explicit timexes (Chambers et al., 2014; Meng et al., 2017). In this work, we address this shortcoming by introducing a framework for including rich rep- resentations of timexes in neural models. These models implicitly capture some information via word embeddings (Mikolov et al., 2013; Penning- ton et al., 2014) or contextualized embeddings such as ELMo (Peters et al., 2018). However, these embeddings do not encode the full richness of temporal information needed for this task. For example, these systems fail to infer the correct event relation …

Content truncated to 500 words