WorldTree V2: A Corpus of Science-Domain Structured Explanations and Inference Patterns supporting Multi-Hop Inference

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 5456–5473 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 5456 WorldTree V2: A Corpus of Science-Domain Structured Explanations and Inference Patterns supporting Multi-Hop Inference Zhengnan Xie‡∗, Sebastian Thiem†∗, Jaycie Martin‡, Elizabeth Wainwright‡, Steven Marmorstein§, Peter Jansen†‡ School of Information†, Department of Linguistics‡, Department of Computer Science§ University of Arizona, Tucson, Arizona, USA pajansen@email.arizona.edu Abstract Explainable question answering for complex questions often requires combining large numbers of facts to answer a question while providing a human-readable explanation for the answer, a process known as multi-hop inference. Standardized science questions require combining an average of 6 facts, and as many as 16 facts, in order to answer and explain, but most existing datasets for multi-hop reasoning focus on combining only two facts, significantly limiting the ability of multi-hop inference algorithms to learn to generate large inferences. In this work we present the second iteration of the WorldTree project, a corpus of 5,114 standardized science exam questions paired with large detailed multi-fact explanations that combine core scientific knowledge and world knowledge. Each explanation is represented as a lexically-connected “explanation graph” that combines an average of 6 facts drawn from a semi-structured knowledge base of 9,216 facts across 66 tables. We use this explanation corpus to author a set of 344 high-level science domain inference patterns similar to semantic frames supporting multi-hop inference. Together, these resources provide training data and instrumentation for developing many-fact multi-hop inference models for question answering. Keywords: question answering, multi-hop inference, explanations 1. Introduction Explainable question answering is the task of providing both answers to natural language questions, as well as detailed human-readable explanations justifying why those answers are correct. Question answering is typically approached using either retrieval or inference methods, where retrieval methods search for a single contiguous passage of text from a corpus or single fact in a knowledge base that provides an answer to a question. For complex questions, a single passage often provides only part of the knowledge required to arrive at a correct answer, and an inference model must combine multiple facts from a corpus or knowledge base to infer the correct answer. In addition to producing an inference that correctly answers a question, a frequent design goal of multi-hop inference algorithms is to use the set of combined facts as a human-readable explanation for why the model’s reasoning is correct. Successfully building multi-hop inference algorithms poses a variety of challenges. Combining facts to perform inference is an inherently noisy process that often drifts off-context to unrelated facts, a phenomenon referred to as semantic drift (Fried et al., 2015). As a result, most multi- hop inference models are generally unable to demonstrate combining more than 2 or 3 facts to perform an inference (Khashabi et al., 2016; Jansen et al., 2017; Das et al., 2017), and even then only infrequently. This is a significant limita- tion, as even the reasoning required to answer elementary science exams averages …

Content truncated to 500 words