Practice Makes a Solver Perfect: Data Augmentation for Math Word Problem Solvers

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4194 - 4206 July 10-15, 2022 ©2022 Association for Computational Linguistics Practice Makes a Solver Perfect: Data Augmentation for Math Word Problem Solvers Vivek Kumar, Rishabh Maheshwary and Vikram Pudi Data Sciences and Analytics Center, Kohli Center on Intelligent Systems International Institute of Information Technology, Hyderabad, India {vivek.k, rishabh.maheshwary}@research.iiit.ac.in, vikram@iiit.ac.in Abstract Existing Math Word Problem (MWP) solvers have achieved high accuracy on benchmark datasets. However, prior works have shown that such solvers do not generalize well and rely on superficial cues to achieve high performance. In this paper, we first conduct experiments to showcase that this behaviour is mainly associ- ated with the limited size and diversity present in existing MWP datasets. Next, we propose several data augmentation techniques broadly categorized into Substitution and Paraphras- ing based methods. By deploying these meth- ods we increase the size of existing datasets by five folds. Extensive experiments on two benchmark datasets across three state-of-the-art MWP solvers shows that proposed methods in- crease the generalization and robustness of ex- isting solvers. On average, proposed methods significantly increase the state-of-the-art results by over five percentage points on benchmark datasets. Further, the solvers trained on the aug- mented dataset performs comparatively better on the challenge test set. We also show the effectiveness of proposed techniques through ablation studies and verify the quality of aug- mented samples through human evaluation. 1 Introduction A Math Word Problem (MWP) consists of natu- ral language text which describes a world state involving some known and unknown quantities, followed by a question text to determine the un- known values. The task is to parse the problem statement and generate equations that can help find the value of unknown quantities. An example of a simple MWP is shown in Table 1. In recent years, the challenge of solving MWP has gained much attention in the NLP community as it needs the development of commonsense multi step reason- ing with numerical quantities. With the rise of deep learning, performance of math solvers has also increased significantly over the years (Wang Original Problem Problem: Nancy grew 8 potatoes. Sandy grew 5 potatoes. How many potatoes did they grow in total ? True Equation: X = 8+5 Paraphrasing Method Problem: How many potatoes did they grow in all given that nancy grew 8 potatoes and sandy grew 5 potatoes. Equation Label: X = 8+5 Substitution Method Problem: Dwight grew 8 potatoes. Juliette grew 5 potatoes. How many potatoes did they grow together ? Equation Label: X = 8+5 Table 1: A MWP and its augmentation examples gen- erated by our methods with preserved equation labels. Blue and Violet colours denote the changes made after the primary stage and secondary stage respectively. et al.; Zhang et al.). However, recent analysis con- ducted in (Kumar et al., 2021) and (Patel et al., 2021) show that these deep learning based solvers rely on shallow heuristics to solve …

Content truncated to 500 words