Neural Math Word Problem Solver with Reinforcement Learning

Proceedings of the 27th International Conference on Computational Linguistics, pages 213–223 Santa Fe, New Mexico, USA, August 20-26, 2018. 213 Neural Math Word Problem Solver with Reinforcement Learning Danqing Huang1∗, Jing Liu2∗, Chin-Yew Lin3, and Jian Yin1 {huangdq2@mail2,issjyin@mail}.sysu.edu.cn liujing46@baidu.com cyl@microsoft.com 1 The School of Data and Computer Science, Sun Yat-sen University. Guangdong Key Laboratory of Big Data Analysis and Processing, Guangzhou, P.R.China 2Baidu Inc. 3Microsoft Research Abstract Sequence-to-sequence model has been applied to solve math word problems. The model takes math problem descriptions as input and generates equations as output. The advantage of sequence-to-sequence model requires no feature engineering and can generate equations that do not exist in training data. However, our experimental analysis reveals that this model suffers from two shortcomings: (1) generate spurious numbers; (2) generate numbers at wrong posi- tions. In this paper, we propose incorporating copy and alignment mechanism to the sequence- to-sequence model (namely CASS) to address these shortcomings. To train our model, we apply reinforcement learning to directly optimize the solution accuracy. It overcomes the “train-test discrepancy” issue of maximum likelihood estimation, which uses the surrogate objective of maximizing equation likelihood during training while the evaluation metric is solution accuracy (non-differentiable) at test time. Furthermore, to explore the effectiveness of our neural model, we use our model output as a feature and incorporate it into the feature-based model. Experi- mental results show that (1) The copy and alignment mechanism is effective to address the two issues; (2) Reinforcement learning leads to better performance than maximum likelihood on this task; (3) Our neural model is complementary to the feature-based model and their combination significantly outperforms the state-of-the-art results. 1 Introduction The task of math word problem solving aims to automatically solve a math problem by reading the text description of the problem and generating the answer. This task requires the machine to have the ability of natural language understanding and reasoning. In the past years, most of the proposed methods heavily rely on predefined rules or feature engineering. On one hand, the rule-based approaches (Bakman, 2007; Liguda and Pfeiffer, 2012; Shi et al., 2015) predefine a structured representation and maps the problem description into the structure by rules. These approaches usually accept only well-formed input and are difficult to scale to other problem types. On the other hand, the feature-based statistical learning approaches (Kushman et al., 2014; Roy and Roth, 2018) generate equation candidates and find the most probable equation by using predefined features. These approaches have two major drawbacks: (1) Their ability of equation generation is weak. An equation is generated by either replacing the numbers of existing equations in the training data, or enumerating possible combinations of math operators, numbers and variables, which leads to intractably huge search space. (2) They need manually designed features that are specific to math word problems. Recent attempts (Ling et al., 2017; Wang et al., 2017) use sequence-to-sequence (seq2seq) model for math word problem solving and they have shown promising results. Wang et al. (2017) apply a …

Content truncated to 500 words