Program Induction by Rationale Generation: Learning to Solve and Explain Algebraic Word Problems

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 158–167 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1015 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 158–167 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1015 Program Induction by Rationale Generation: Learning to Solve and Explain Algebraic Word Problems Wang Ling♠ Dani Yogatama♠ Chris Dyer♠ Phil Blunsom♠♦ ♠DeepMind ♦University of Oxford {lingwang,dyogatama,cdyer,pblunsom}@google.com Abstract Solving algebraic word problems re- quires executing a series of arithmetic operations—a program—to obtain a final answer. However, since programs can be arbitrarily complicated, inducing them directly from question-answer pairs is a formidable challenge. To make this task more feasible, we solve these problems by generating answer rationales, sequences of natural language and human-readable mathematical expressions that derive the final answer through a series of small steps. Although rationales do not explic- itly specify programs, they provide a scaf- folding for their structure via intermedi- ate milestones. To evaluate our approach, we have created a new 100,000-sample dataset of questions, answers and ratio- nales. Experimental results show that in- direct supervision of program learning via answer rationales is a promising strategy for inducing arithmetic programs. 1 Introduction Behaving intelligently often requires mathemat- ical reasoning. Shopkeepers calculate change, tax, and sale prices; agriculturists calculate the proper amounts of fertilizers, pesticides, and wa- ter for their crops; and managers analyze produc- tivity. Even determining whether you have enough money to pay for a list of items requires applying addition, multiplication, and comparison. Solv- ing these tasks is challenging as it involves rec- ognizing how goals, entities, and quantities in the real-world map onto a mathematical formaliza- tion, computing the solution, and mapping the so- lution back onto the world. As a proxy for the richness of the real world, a series of papers have used natural language specifications of algebraic word problems, and solved these by either learn- ing to fill in templates that can be solved with equation solvers (Hosseini et al., 2014; Kushman et al., 2014) or inferring and modeling operation sequences (programs) that lead to the final an- swer (Roy and Roth, 2015). In this paper, we learn to solve algebraic word problems by inducing and modeling programs that generate not only the answer, but an answer ratio- nale, a natural language explanation interspersed with algebraic expressions justifying the overall solution. Such rationales are what examiners re- quire from students in order to demonstrate un- derstanding of the problem solution; they play the very same role in our task. Not only do natural language rationales enhance model interpretabil- ity, but they provide a coarse guide to the structure of the arithmetic programs that must be executed. In fact the learner we propose (which relies on a heuristic search; §4) fails to solve this task with- out modeling the rationales—the search space is too unconstrained. This work is thus related to models that can explain or rationalize …

Content truncated to 500 words