Temporal Knowledge Base Completion: New Algorithms and Evaluation Protocols

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 3733–3747, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 3733 Temporal Knowledge Base Completion: New Algorithms and Evaluation Protocols Prachi Jain∗1, Sushant Rathi∗1, Mausam1 and Soumen Chakrabarti2 1 Indian Institute of Technology Delhi 2 Indian Institute of Technology Bombay {p6.jain, rathisushant5}@gmail.com, mausam@cse.iitd.ac.in, soumen.chakrabarti@gmail.com Abstract Research on temporal knowledge bases, which associate a relational fact (s, r, o) with a va- lidity time period (or time instant), is in its early days. Our work considers predicting missing entities (link prediction) and missing time intervals (time prediction) as joint Tem- poral Knowledge Base Completion (TKBC) tasks, and presents TIMEPLEX, a novel TKBC method, in which entities, relations and, time are all embedded in a uniform, compatible space. TIMEPLEX exploits the recurrent na- ture of some facts/events and temporal interac- tions between pairs of relations, yielding state- of-the-art results on both prediction tasks. We also find that existing TKBC models heav- ily overestimate link prediction performance due to imperfect evaluation mechanisms. In response, we propose improved TKBC evalua- tion protocols for both link and time prediction tasks, dealing with subtle issues that arise from the partial overlap of time intervals in gold in- stances and system predictions. 1 Introduction A knowledge base (KB) is a collection of triples (s, r, o), with a subject s, a relation type r and an object o. KBs are usually incomplete, necessitating completion (KBC), i.e., inferring facts not provided in the KB. A KBC model is often evaluated via link prediction: supplying missing arguments to queries of the form (s, r, ?) and (?, r, o). Many relations are transient or impermanent. Temporal KBs annotate each fact (event) with the time period (instant) in which it holds (occurs) (Hoffart et al., 2013). A person is born in a city in an instant, a politician can be a country’s president for several years, and a marriage may last between years and decades. Temporal KBs represent these by (s, r, o, T) tuples, where T is a span of time. ∗ Equal contribution Temporal KBC (TKBC) performs completion of temporal KBs. It is also primarily evaluated by link prediction queries (s, r, ?, T) and (?, r, o, T). Recently, time prediction (s, r, o, ?) has also been considered for predicting time instants, but not time intervals (Lacroix et al., 2020). While KBC has been intensely researched, TKBC is only beginning to be explored. TKBC presents novel challenges in task definition and modeling. For instance, little is known about how best to predict intervals for (s, r, o, ?) queries, or how to evaluate a system response interval. More- over, we show that even for link prediction queries, evaluation faces subtle complications owing to the inclusion of T in (s, r, ?, T) queries and requires careful rethinking of evaluation protocols. In this paper, we propose improved evaluation protocols for both link and time prediction tasks in a TKBC. TKBC also brings unique modeling opportuni- …

Content truncated to 500 words