MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms

Proceedings of NAACL-HLT 2019, pages 2357–2367 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2357 MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms Aida Amini1, Saadia Gabriel1, Shanchuan Lin1, Rik Koncel-Kedziorski1, Yejin Choi1,2, and Hannaneh Hajishirzi1,2 1University of Washington 2Allen Institute for AI {amini91, skgabrie, linsh, kedzior, yejin, hannaneh}@cs.washington.edu Abstract We introduce a large-scale dataset of math word problems and an interpretable neural math problem solver that learns to map prob- lems to operation programs. Due to an- notation challenges, current datasets in this domain have been either relatively small in scale or did not offer precise operational an- notations over diverse problem types. We introduce a new representation language to model precise operation programs correspond- ing to each math problem that aim to im- prove both the performance and the inter- pretability of the learned models. Using this representation language, our new dataset, MathQA, significantly enhances the AQuA dataset with fully-specified operational pro- grams. We additionally introduce a neu- ral sequence-to-program model enhanced with automatic problem categorization. Our exper- iments show improvements over competitive baselines in our MathQA as well as the AQuA datasets. The results are still significantly lower than human performance indicating that the dataset poses new challenges for future re- search. Our dataset is available at: https: //math-qa.github.io/math-QA/. 1 Introduction Answering math word problems poses unique challenges for logical reasoning over implicit or explicit quantities expressed in text. Math word- problem solving requires extraction of salient in- formation from natural language narratives. Auto- matic solvers must transform the textual narratives into executable meaning representations, a process that requires both high precision and, in the case of story problems, significant world knowledge. As shown by the geometry question in Figure 1, math word problems are generally narratives de- scribing the progress of actions and relations over some entities and quantities. The operation pro- An artist wishes to paint a circular region on a square poster that is 3.4 feet on a side. if the area of the circular region is to be 1/2 the area of the poster, what must be the radius of the circular region in feet? Math word problem Square_area(3.4) Operation 1 Multiply(11.56, 0.5) Operation 2 Divide(5.78, const_pi) Operation 3 Sqrt(1.8343) Operation 4 3.4 0.5 3.4 0.5 11.56 3.4, 0.5, 11.56, 5.78 Output == 1.3543 3.4 0.5 11.56 5.78 1.8343 Figure 1: Example of a math word problem aligned with representation language by crowd-sourced anno- tation gram underlying the problem in Figure 1 high- lights the complexity of the problem-solving task. Here, we need the ability to deduce implied con- stants (pi) and knowledge of domain-specific for- mulas (area of the square). In this paper, we introduce a new operation- based representation language for solving math word problems. We use this representation lan- guage to construct MathQA1, a new large-scale, diverse dataset of 37k English multiple-choice math word problems covering multiple math do- main categories by modeling operation programs corresponding to word problems in the AQuA dataset …

Content truncated to 500 words