Answering Elementary Science Questions by Constructing Coherent Scenes using Background Knowledge

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2007–2012, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Answering Elementary Science Questions by Constructing Coherent Scenes using Background Knowledge Yang Li∗ UC Santa Barbara Santa Barbara, CA, USA yangli@cs.ucsb.edu Peter Clark Allen Institute for Artificial Intelligence Seattle, WA, USA peterc@allenai.org Abstract Much of what we understand from text is not explicitly stated. Rather, the reader uses his/her knowledge to fill in gaps and create a coherent, mental picture or “scene” depicting what text appears to convey. The scene constitutes an under- standing of the text, and can be used to an- swer questions that go beyond the text. Our goal is to answer elementary science questions, where this requirement is per- vasive; A question will often give a partial description of a scene and ask the student about implicit information. We show that by using a simple “knowledge graph” rep- resentation of the question, we can lever- age several large-scale linguistic resources to provide missing background knowl- edge, somewhat alleviating the knowledge bottleneck in previous approaches. The coherence of the best resulting scene, built from a question/answer-candidate pair, re- flects the confidence that the answer can- didate is correct, and thus can be used to answer multiple choice questions. Our ex- periments show that this approach outper- forms competitive algorithms on several datasets tested. The significance of this work is thus to show that a simple “knowl- edge graph” representation allows a ver- sion of “interpretation as scene construc- tion” to be made viable. 1 Introduction Elementary grade science tests are challenging as they test a wide variety of commonsense knowl- edge that human beings largely take for granted, yet are very difficult for machines (Clark, 2015). For example, consider a question from a NY Re- gents 4th Grade science test: ∗Work was done while the author was an intern at Allen Institute for Artificial Intelligence. Question 1 “When a baby shakes a rattle, it makes a noise. Which form of energy was changed to sound energy?” [Answer: mechanical energy] Science questions are typically quite different from the entity-centric factoid questions exten- sively studied in the question answering (QA) community, e.g., “In which year was Bill Clinton born?” (Ferrucci et al., 2010; Yao and Van Durme, 2014). While factoid questions are usually an- swerable from text search or fact databases, sci- ence questions typically require deeper analysis. A full understanding of the above question in- volves not just parsing and semantic interpreta- tion; it involves adding implicit information to cre- ate an overall picture of the “scene” that the text is intended to convey, including facts such as: noise is a kind of sound, the baby is holding the rattle, shaking involves movement, the rattle is making the noise, movement involves mechanical energy, etc. This mental ability to create a scene from par- tial information is at the heart of natural language understanding (NLU), which is essential for an- swering these kinds of question. …

Content truncated to 500 words