Temporal Common Sense Acquisition with Minimal Supervision

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7579–7589 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 7579 Temporal Common Sense Acquisition with Minimal Supervision Ben Zhou,1 Qiang Ning,2 Daniel Khashabi,2 Dan Roth1 1University of Pennsylvania, 2Allen Institute for AI {xyzhou,danroth}@cis.upenn.edu {qiangn,danielk}@allenai.org Abstract Temporal common sense (e.g., duration and frequency of events) is crucial for understand- ing natural language. However, its acquisi- tion is challenging, partly because such in- formation is often not expressed explicitly in text, and human annotation on such concepts is costly. This work proposes a novel se- quence modeling approach that exploits ex- plicit and implicit mentions of temporal com- mon sense, extracted from a large corpus, to build TACOLM,1 a temporal common sense language model. Our method is shown to give quality predictions of various dimen- sions of temporal common sense (on UDST and a newly collected dataset from Real- News). It also produces representations of events for relevant tasks such as duration com- parison, parent-child relations, event corefer- ence and temporal QA (on TimeBank, HiEVE and MCTACO) that are better than using the standard BERT. Thus, it will be an important component of temporal NLP. 1 Introduction Time is crucial when describing the evolving world. It is thus important to understand time as expressed in natural language text. Indeed, many natural lan- guage understanding (NLU) applications, includ- ing information retrieval, summarization, causal inference, and QA (UzZaman et al., 2013; Cham- bers et al., 2014; Llorens et al., 2015; Bethard et al., 2016; Leeuwenberg and Moens, 2017; Ning et al., 2018b), rely on understanding time. However, understanding time in natural lan- guage text heavily relies on common sense infer- ence. Such inference is challenging since common- sense information is rarely made explicit in text (e.g., how long does it take to open a door?) Even when such information is mentioned, it is often 1https://cogcomp.seas.upenn.edu/page/ publication_view/904 0 0.1 0.2 0.3 0.4 seconds hour week year Dr. Porter is taking a walk. Dr. Porter is taking his weekend break. Dr. Porter is taking a long vacation. 0 0.1 0.2 0.3 seconds hour week year 0 0.1 0.2 0.3 0.4 seconds hour week year Dr. Porter is taking a walk. Dr. Porter is taking his weekend break. Dr. Porter is taking a long vacation. Frequency 0 0.1 0.2 0.3 seconds hour week year walk weekend vacation vacation weekend walk Duration Figure 1: Our model’s predicted distributions of event duration and frequency. The model is able to attend to contextual information and thus produce reasonable estimates. affected by another type of reporting bias: people rarely say the obvious, in order to communicate more efficiently, but sometimes highlight rarities (Schubert, 2002; Van Durme, 2009; Gordon and Van Durme, 2013; Zhang et al., 2017; Bauer et al., 2018; Tandon et al., 2018). This is an even more pronounced phenomenon when it comes to temporal common sense (TCS) (Zhou et al., 2019). In Example 1, human read- ers know that a typical vacation is likely …

Content truncated to 500 words