Interpretable Math Word Problem Solution Generation via Step-by-step Planning

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 6858–6877 July 9-14, 2023 ©2023 Association for Computational Linguistics Interpretable Math Word Problem Solution Generation Via Step-by-step Planning Mengxue Zhang1, Zichao Wang2, Zhichao Yang1, Weiqi Feng1, Andrew Lan1 1University of Massachusetts Amherst, 2Adobe Research mengxuezhang@umass.edu Abstract Solutions to math word problems (MWPs) with step-by-step explanations are valuable, espe- cially in education, to help students better com- prehend problem-solving strategies. Most ex- isting approaches only focus on obtaining the final correct answer. A few recent approaches leverage intermediate solution steps to improve final answer correctness but often cannot gen- erate coherent steps with a clear solution strat- egy. Contrary to existing work, we focus on improving the correctness and coherence of the intermediate solutions steps. We propose a step-by-step planning approach for interme- diate solution generation, which strategically plans the generation of the next solution step based on the MWP and the previous solution steps. Our approach first plans the next step by predicting the necessary math operation needed to proceed, given history steps, then generates the next step, token-by-token, by prompting a language model with the predicted math op- eration. Experiments on the GSM8K dataset demonstrate that our approach improves the ac- curacy and interpretability of the solution on both automatic metrics and human evaluation. 1 Introduction Arithmetic math word problems (MWPs) consist of natural language statements describing real-world scenarios that involve numerical quantities, fol- lowed by a question asking for an unknown value. Solving MWPs require parsing the textual state- ments and carrying out the corresponding calcula- tions (Kumar et al., 2022). MWPs are an important educational tool that helps assess and improve stu- dent knowledge in basic mathematical concepts and skills (Walkington, 2013; Verschaffel et al., 2020). They also represent a long-standing interest in artificial intelligence (AI) research since cor- rectly solving them serves as a key benchmark task for testing and improving the mathematical reason- ing skills of AI models (Feigenbaum and Feldman, 1995; Bommasani et al., 2021; Cobbe et al., 2021; Lewkowycz et al., 2022). There is a large body of literature that focuses on automatically solving MWP. Earlier works took a modular approach that first analyzes unconstrained natural language and then maps intricate text pat- terns onto mathematical vocabulary (Sundaram et al., 2022). As a result, this approach relies heav- ily on hand-crafted rules to fill the gap between natural language and symbolic mathematical vo- cabulary (Sundaram et al., 2022). Recent works leverage advances in natural language processing and take a neural network-based, end-to-end ap- proach, where a neural network encodes a numeri- cal representation of the MWP (and the underlying equation), from which a decoder generates the final answer (Zou and Lu, 2019; Wang et al., 2017; Wu et al., 2020; Chen et al., 2020; Cao et al., 2021; Shen et al., 2021; Shao et al., 2022; Jie et al., 2022). Unfortunately, the vast majority of these works focus on generating and predicting a single final answer, …

Content truncated to 500 words