From Textbooks to Knowledge: A Case Study in Harvesting Axiomatic Knowledge from Textbooks to Solve Geometry Problems

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 773–784 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics From Textbooks to Knowledge: A Case Study in Harvesting Axiomatic Knowledge from Textbooks to Solve Geometry Problems Mrinmaya Sachan Avinava Dubey Eric P. Xing School of Computer Science Carnegie Mellon University {mrinmays, akdubey, epxing}@cs.cmu.edu Abstract Textbooks are rich sources of knowledge. Harvesting knowledge from textbooks is a key challenge in many educational appli- cations. In this paper, we present an ap- proach to obtain axiomatic knowledge of geometry in the form of horn-clause rules from math textbooks. The approach uses rich contextual and typographical features extracted from the textbooks. It also lever- ages the redundancy and shared ordering of axioms across multiple textbooks to ac- curately harvest axioms. These axioms are then parsed into horn-clause rules that are used to improve the state-of-the-art in solving geometry problems. 1 Introduction Recently, researchers have proposed standardized tests as “drivers for progress in AI” (Clark and Etzioni, 2016). There is a growing body of work in solving standardized tests such as reading comprehensions (Richardson et al., 2013; Sachan et al., 2015, inter alia), science question answering (Schoenick et al., 2016; Sachan et al., 2016, in- ter alia), algebra word problems (Kushman et al., 2014, inter alia), geometry problems (Seo et al., 2015), pre-university entrance exams (Fujita et al., 2014), etc. A major challenge in building these solvers is the lack of subject knowledge. For ex- ample, geometry tests require knowledge of ge- ometry axioms and pre-university exams require knowledge of laws of physics, chemistry, etc. In this paper, we present an automatic approach that can (a) harvest such subject knowledge from textbooks, and (b) parse the extracted knowledge to structured programs that the solvers can use. Unlike information extraction systems trained on domains such as web documents (Chang et al., Figure 1: An excerpt of a textbook from our dataset that introduces the Pythagoras theorem. The textbook has a lot of typographical features that can be used to harvest this theorem: The textbook explicitly labels it as a “the- orem”; there is a colored bounding box around it; an equation writes down the rule and there is a supporting figure. Our models leverages such rich con- textual and typographical information (when available) to accurately harvest axioms and then parses them to horn-clause rules. The horn-clause rule de- rived by our approach for the Pythagoras theorem is: isT riangle(ABC) ∧ perpendicular(AC, BC) =⇒ BC2 + AC2 = AB2. 2003; Etzioni et al., 2004, inter alia), learning an information extraction system that can extract ax- iomatic knowledge from textbooks is challenging because of the small amount of in-domain labeled data available for these tasks. We tackle this chal- lenge by (a) leveraging the redundancy and shared ordering of axiom mentions across multiple text- books1, and (b) utilizing rich contextual and typo- graphical features2 from textbooks to effectively extract and parse axioms. Finally, we also provide an approach to parse the extracted …

Content truncated to 500 words