Generic Temporal Reasoning with Differential Analysis and Explanation

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 12013–12029 July 9-14, 2023 ©2023 Association for Computational Linguistics Generic Temporal Reasoning with Differential Analysis and Explanation Yu Feng, Ben Zhou, Haoyu Wang, Helen Jin, Dan Roth University of Pennsylvania {fengyu1, xyzhou, why16gzl, helenjin, danroth}@seas.upenn.edu Abstract Temporal reasoning is the task of predicting temporal relations of event pairs. While tem- poral reasoning models can perform reason- ably well on in-domain benchmarks, we have little idea of these systems’ generalizability due to existing datasets’ limitations. In this work, we introduce a novel task named TODAY that bridges this gap with temporal differential analysis, which as the name suggests, evalu- ates whether systems can correctly understand the effect of incremental changes. Specifically, TODAY introduces slight contextual changes for given event pairs, and systems are asked to tell how this subtle contextual change would affect relevant temporal relation distributions. To facilitate learning, TODAY also annotates human explanations. We show that existing models, including GPT-3.5, drop to random guessing on TODAY, suggesting that they heav- ily rely on spurious information rather than proper reasoning for temporal predictions. On the other hand, we show that TODAY’s super- vision style and explanation annotations can be used in joint learning, encouraging models to use more appropriate signals during training and thus outperform across several benchmarks. TODAY can also be used to train models to so- licit incidental supervision from noisy sources such as GPT-3.5, thus moving us more toward the goal of generic temporal reasoning systems. 1 Introduction Temporal relation extraction (Pustejovsky et al., 2003; Chambers et al., 2014) is traditionally viewed as an information extraction task, where a model uses explicit temporal signals such as “happened before” to identify the temporal order of events. While these models have contributed to many downstream pipelines, they are not enough for more complicated tasks such as timeline gener- ation, where most event pairs do not come with explicit signals. These implicit temporal relation extractions (Zhou et al., 2021) thus require tempo- Figure 1: A morning and coffee shop scenario exam- ple of temporal differential analysis. When adding the Additional Sentence 1 to the context, the temporal re- lation between the pair of events shifts towards before. Meanwhile, when adding the Additional Sentence 2, the relation shifts towards after. ral reasoning, which relies on both common sense and semantic understanding of the context. In re- cent works, a popular approach to address these predictions is to finetune pre-trained language mod- els (PLMs) with annotated supervision data. Un- fortunately, existing temporal benchmarks (Puste- jovsky et al., 2003; Cassidy et al., 2014; Ning et al., 2018a) only annotate hard labels and ignore the fact that temporal labels can often be soft and nondeter- ministic. This approach allows models to exploit spurious signals and annotation artifacts easily for performance. For example, a model may learn to predict “lunch” before “dinner” regardless of the surrounding context, yet most existing benchmarks will not challenge such beliefs because most …

Content truncated to 500 words