Weakly Supervised Formula Learner for Solving Mathematical Problems

Proceedings of the 29th International Conference on Computational Linguistics, pages 1743–1752 October 12–17, 2022. 1743 Weakly Supervised Formula Learner for Solving Mathematical Problems Yuxuan Wu and Hideki Nakayama The University of Tokyo {wuyuxuan,nakayama}@nlab.ci.i.u-tokyo.ac.jp Abstract Mathematical reasoning task is a subset of the natural language question answering task. Ex- isting work suggested solving this task with a two-phase approach, where the model first predicts formulas from questions and then cal- culates answers from such formulas. This ap- proach achieved desirable performance in ex- isting work. However, its reliance on anno- tated formulas as intermediate labels through- out its training limited its application. In this work, we put forward the idea to en- able models to learn optimal formulas au- tonomously. We proposed Weakly Supervised Formula Learner, a learning framework that drives the formula exploration with weak su- pervision from the final answers to mathemat- ical problems. Our experiments are conducted on two representative mathematical reasoning datasets MathQA and Math23K. On MathQA, our method outperformed baselines trained on complete yet imperfect formula annotations. On Math23K, our method outperformed other weakly supervised learning methods. 1 1 Introduction Mathematical reasoning is a task where mathemat- ical problems are described in natural language or mathematical symbols. Such problems require values, expressions, or other mathematical repre- sentations as answers. A naive approach to solving this task is to treat it as an end-to-end token-by- token predicting problem from questions to an- swers. However, this approach showed a rela- tively poor generalization capacity on unseen num- bers (Saxton et al., 2019). Another approach to solving mathematical reasoning task is to adopt a two-phase methodology. In the first phase, specific formulas are predicted for each question. In the second phase, such formulas are calculated under predefined rules to produce the final answers. This 1The software is available at https://github.com/ evan-ak/wsfl. approach has been widely applied in recent work and has achieved desirable results in many repre- sentative mathematical reasoning datasets (Wang et al., 2017; Amini et al., 2019). However, this two-phase solution leads to a reliance on annotated formulas as indispensable labels for training the formula predictor in the first phase. This reliance further results in two major weaknesses. Firstly, ground-truth formula annotations are not necessar- ily prepared for every mathematical problem and dataset. This makes it impossible to extend this solution to datasets without these annotations. Sec- ondly, the learning process can be misled when there is noise in the formula annotations. In con- sideration of this, we are motivated to propose a new learning framework for solving mathematical problems that is not dependent on formula anno- tations. On the whole, we followed the principle of the two-phase methodology and implemented the two phases with what we call PolicyNet and ActTaker. Figure 1 is an overview of our learning framework. Our main contributions in this work can be summarized as follows: • We established a new mechanism to learn for- mulas with weak supervision from final answers, which outperforms existing weakly supervised learning methods. • We enabled …

Content truncated to 500 words