DocTime: A Document-level Temporal Dependency Graph Parser

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 993 - 1009 July 10-15, 2022 ©2022 Association for Computational Linguistics DocTime: A Document-level Temporal Dependency Graph Parser Puneet Mathur‡, Vlad I Morariu†, Verena Kaynig-Fittkau†, Jiuxiang Gu†, Franck Dernoncourt†, Quan Hung Tran†, Ani Nenkova†,Dinesh Manocha‡, Rajiv Jain† ‡ University of Maryland, College Park † Adobe Research ‡{puneetm,dmanocha}@umd.edu ‡{morariu,kaynigfi,jigu,dernonco}@adobe.com ‡{qtran,nenkova,rajijain}@adobe.com Abstract We introduce DocTime - a novel temporal dependency graph (TDG) parser that takes as input a text document and produces a tempo- ral dependency graph. It outperforms previous BERT based solutions by a relative 4-8% on three datasets from modeling the problem as a graph-network with path-prediction loss to incorporate longer range dependencies. This work also demonstrates how the TDG graph can be used to improve the downstream tasks of temporal questions answering and NLI by a relative 4-10% with a new framework that in- corporates the temporal dependency graph into the self-attention layer of Transformer models (Time-transformer). Finally, we develop and evaluate on a new temporal dependency graph dataset for the domain of contractual documents, which has not been previously ex- plored in this setting. 1 Introduction Understanding the temporal relations between events mentioned in a document is an important nat- ural language task with applications in downstream tasks such as timeline creation (Leeuwenberg and Moens, 2018), time-aware summarization (Noh et al., 2020), temporal question-answering (Ning et al., 2020a), and temporal information extraction (Leeuwenberg and Moens, 2019). This area of re- search remains important yet challenging due to several limitations such as confounded modalities (eg. events that are certain to happen vs the ones that might happen), event ambiguity (eg. agreeing to terms of a contract vs signing a contract) and need for complete annotation of all event pairs for precise temporal localization (Yao et al., 2020a). Early work densely annotated all pairs of events to address this problem (Cassidy et al., 2014), but was limited to short passages or adjacent sentences due to the �n 2 � complexity of the task, especially for long documents. Recently this problem formu- lation was significantly simplified using temporal dependency trees (TDT) (Zhang and Xue, 2019) and temporal dependency graphs (TDG) (Yao et al., 2020a) by only capturing the reference TIMEX or event to build a dependency graph to capture this information. This enabled the development of tem- poral dependency parsers (Zhang and Xue, 2018a; Ross et al., 2020a) to infer temporal relationships more robustly and efficiently. We introduce DocTime - a state-of-the-art tem- poral dependency parser that parses document-level text to produce temporal dependency graphs. Un- like previous approaches using contextual features such as BERT(Ross et al., 2020b), our model uti- lizes a graph network and a novel path prediction loss to reason over long-range multi-hop depen- dencies while maintaining global consistency of temporal ordering of inter-dependent events. To validate the utility of DocTime and our generated temporal dependency graph, we go one step further than prior work and explore the …

Content truncated to 500 words