GraphMR: Graph Neural Network for Mathematical Reasoning

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3395–3404 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 3395 GraphMR: Graph Neural Network for Mathematical Reasoning Weijie Feng†, Binbin Liu†, Dongpeng Xu∗, Qilong Zheng†, Yun Xu† †University of Science and Technology of China ∗University of New Hampshire {fengwj, robbertl}@mail.ustc.edu.cn dongpeng.xu@unh.edu {qlzheng, xuyun}@ustc.edu.cn Abstract Mathematical reasoning aims to infer satisfi- able solutions based on the given mathemat- ics questions. Previous natural language pro- cessing researches have proven the effective- ness of sequence-to-sequence (Seq2Seq) or re- lated variants on mathematics solving. How- ever, few works have been able to explore structural or syntactic information hidden in expressions (e.g., precedence and associativ- ity). This dissertation set out to investigate the usefulness of such untapped information for neural architectures. Firstly, mathemati- cal questions are represented in the format of graphs within syntax analysis. The structured nature of graphs allows them to represent rela- tions of variables or operators while preserv- ing the semantics of the expressions. Hav- ing transformed to the new representations, we proposed a graph-to-sequence neural network GraphMR, which can effectively learn the hier- archical information of graphs inputs to solve mathematics and speculate answers. A com- plete experimental scenario with four classes of mathematical tasks and three Seq2Seq base- lines is built to conduct a comprehensive anal- ysis, and results show that GraphMR outper- forms others in hidden information learning and mathematics resolving. 1 Introduction Mathematical reasoning aims to infer satisfactory solutions for the given mathematical problem based on logical rules. It plays a pivotal role in com- puter algebra (Risch, 1970; Bronstein, 2005; Ged- des et al., 1992) (e.g., automated theorem prov- ing), design formulas in numerical programs, and complete scientific computations. With the cre- ation of massive data of logical expressions, neu- ral networks (NNs) based on statistical learning (Rumelhart and McClelland, 1986) have shown success on mathematical reasoning tasks countered to rule-based approaches. For example, Evans et al. (2018); Allamanis et al. (2017) applied tree neural networks to process logic entailment and boolean arithmetic problems. (Lample and Charton, 2019; Piotrowski et al., 2019) released elaborated NNs for symbolic integration and differential equations, shown greater competitiveness than algebra sys- tems. However, much of the sequence-to-sequence (Seq2Seq) methods up to now (Kushman et al., 2014; Ling et al., 2017; Piotrowski et al., 2019; Sax- ton et al., 2019) have not explored the structural in- formation behind expressions, such as precedence and associativity. For example, modern Seq2Seq models treat structured symbolic expressions with specific syntax as serialized natural language sen- tences, which result treat x+y and y+x as different expressions. Moreover, most of these works focus on single tasks (Kaiser and Sutskever, 2016; Trask et al., 2018), lack the capability to handle complex computation. This study seeks to excavate the implicit informa- tion hidden in mathematical problems and examine the capacities of graph-to-sequence architecture to solve large-scope mathematical problems, such as algebra, arithmetic, and polynomial. The first step in this study was to investigate efficient graph substitutes …

Content truncated to 500 words