Learning to Solve Geometry Problems from Natural Language Demonstrations in Textbooks

Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017), pages 251–261, Vancouver, Canada, August 3-4, 2017. c⃝2017 Association for Computational Linguistics Learning to Solve Geometry Problems from Natural Language Demonstrations in Textbooks Mrinmaya Sachan Eric P. Xing School of Computer Science Carnegie Mellon University {mrinmays, epxing}@cs.cmu.edu Abstract Humans as well as animals are good at imitation. Inspired by this, the learning by demonstration view of machine learn- ing learns to perform a task from detailed example demonstrations. In this paper, we introduce the task of question answer- ing using natural language demonstra- tions where the question answering system is provided with detailed demonstrative solutions to questions in natural language. As a case study, we explore the task of learning to solve geometry problems using demonstrative solutions available in text- books. We collect a new dataset of demon- strative geometry solutions from textbooks and explore approaches that learn to in- terpret these demonstrations as well as to use these interpretations to solve geometry problems. Our approaches show improve- ments over the best previously published system for solving geometry problems. 1 Introduction Cognitive science emphasizes the importance of imitation or learning by example (Meltzoff and Moore, 1977; Meltzoff, 1995) in human learn- ing. When a teacher signals a pedagogical inten- tion, children tend to imitate the teacher’s actions (Buchsbaum et al., 2011; Butler and Markman, 2014). Inspired by this phenomenon, the learn- ing by demonstration view of machine learning (Schaal, 1997; Argall et al., 2009; Goldwasser and Roth, 2014) assumes training data in the form of example demonstrations. A task is demonstrated by a teacher and the learner generalizes from these demonstrations in order to execute the task. In this paper, we introduce the novel task of question answering using natural language Text Description: measure( MAO, 30o) isCircle(O) radius(O, 4 cm) ?x Diagram: liesOn( A, circle O), liesOn( B, circle O), liesOn( C, circle O), liesOn( D, circle O) isLine(AB), isLine(BC), isLine(CA), isLine(BD), isLine(DA) isTriangle(ABC), isTriangle(ABD), isTriangle(AOM) measure( ADB, x), measure( MAO, 30o) measure( AMO, 90o) … Figure 1: Above: An example SAT style geometry problem with the text description, corresponding diagram and (option- ally) answer candidates. Below: A logical expression that represents the meaning of the text description and the dia- gram in the problem. GEOS derives a weighted logical ex- pression where each predicates also carry a weighted score but we do not show them here for clarity. demonstrations. Research in question answer- ing has traditionally focused on learning from question-answer pairs (Burger et al., 2001). How- ever, it is well-established in the educational psy- chology literature (Allington and Cunningham, 2010; Felder et al., 2000) that children tend to learn better and faster from concrete illustrations and demonstrations. In this paper, we raise the question – “Can we leverage demonstrative solu- tions for questions as provided by a teacher to im- prove our question answering systems?” As a case study, we propose the task of learn- ing to solve SAT geometry problems (such as the one …

Content truncated to 500 words