MAWPS: A Math Word Problem Repository

Proceedings of NAACL-HLT 2016, pages 1152–1157, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics MAWPS: A Math Word Problem Repository Rik Koncel-Kedziorski*1, Subhro Roy*2, Aida Amini1, Nate Kushman3, and Hannaneh Hajishirzi1 1University of Washington, 2University of Illinois Urbana Champaign, 3Microsoft Research {kedzior, amini91, hannaneh}@uw.edu, sroy9@illinois.edu, nkushman@microsoft.com Abstract Recent work across several AI subdisciplines has focused on automatically solving math word problems. In this paper we introduce MAWPS, an online repository of Math Word Problems, to provide a unified testbed to eval- uate different algorithms. MAWPS allows for the automatic construction of datasets with particular characteristics, providing tools for tuning the lexical and template overlap of a dataset as well as for filtering ungrammatical problems from web-sourced corpora. The on- line nature of this repository facilitates easy community contribution. At present, we have amassed 3,320 problems, including the full datasets used in several prominent works. 1 Introduction Automatically solving math word problems has proved a difficult and interesting challenge for the AI research community (Feigenbaum et al., 1995). Math word problems serve as a testbed for algo- rithms that build a precise understanding of what is being asserted in text. Consider the following: Rachel bought two coloring books. One had 23 pictures and the other had 32. After one week she had colored 44 of the pictures. How many pictures does she still have to color? To solve such a problem, an algorithm must model implicit and explicit quantities in the text and their relationships through mathematical operations. Many researchers have taken on this challenge to design data-driven approaches to solve different * denotes equal contribution types of math word problems (Hosseini et al., 2014; Kushman et al., 2014; Roy and Roth, 2015; Zhou et al., 2015; Koncel-Kedziorski et al., 2015). Treat- ments of this task range from template-matching to narrative-building, and methods including integer linear programming, factorization, and more appear in the literature. As a result of the variety of ap- proaches, several datasets have emerged. The va- riety of math word problems in these datasets is such that researchers have already begun specializ- ing in the types of problems that their systems are able to successfully solve. Additionally, in some datasets one may find significant repetition of com- mon words, a small set of equation templates used again and again, or problem texts which map to highly degraded grammatical structures. Therefore, designing general algorithms that can address dif- ferent problem types while still being robust to the variations across datasets has remained a challenge. In response to the burgeoning interest in this task, we present MAWPS (MAth Word ProblemS, pro- nounced mops), a framework for building an on- line repository of math word problems. Our frame- work includes a collection of varying types of math word problems, their answers, and equation tem- plates, as well as interfaces which allow researchers to dynamically update and add more problem types. Additionally, in light of the problematic features of the current datasets cited above, our framework pro- vides the …

Content truncated to 500 words