Modeling Intra-Relation in Math Word Problems with Different Functional Multi-Head Attentions

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6162–6167 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 6162 Modeling Intra-Relation in Math Word Problems with Different Functional Multi-Head Attentions Jierui Li1, Lei Wang1∗, Jipeng Zhang1, Yan Wang2, Bing Tian Dai3, Dongxiang Zhang45 1Center for Future Media and School of Computer Science & Engineering, UESTC, 2Tencent AI Lab 3School of Information Systems, Singapore Management University, 4Afanti Research, 5Zhejiang University {lijierui, zhangjipeng20}@std.uestc.edu.cn, demolei@outlook.com bradenwang@tencent.com, btdai@smu.edu.sg, zhangdongxiang37@gmail.com Abstract Several deep learning models have been proposed for solving math word problems (MWPs) automatically. Although these mod- els have the ability to capture features with- out manual efforts, their approaches to cap- turing features are not specifically designed for MWPs. To utilize the merits of deep learning models with simultaneous consider- ation of MWPs’ specific features, we propose a group attention mechanism to extract global features, quantity-related features, quantity- pair features and question-related features in MWPs respectively. The experimental results show that the proposed approach performs sig- nificantly better than previous state-of-the-art methods, and boost performance from 66.9% to 69.5% on Math23K with training-test split, from 65.8% to 66.9% on Math23K with 5-fold cross-validation and from 69.2% to 76.1% on MAWPS. 1 Introduction Computer systems, dating back to 1960s, have been developing to automatically solve math word problems (MWPs) (Feigenbaum and Feldman, 1963; Bobrow, 1964). As illustrated in Table 1, when solving this problem, machines are asked to infer “how many shelves would Tom fill up ” based on the textual problem description. It requires sys- tems having the ability to map the natural language text into the machine-understandable form, reason in terms of sets of numbers or unknown variables, and then derive the numeric answer. In recent years, a growing number of deep learning models for MWPs (Wang et al., 2017; Ling et al., 2017; Wang et al., 2018b,a; Huang et al., 2018a,b; Wang et al., 2019) have drawn inspiration from advances in machine translation. ∗ corresponding author Problem: For a birthday party Tom bought 4 regular sodas and 52 diet sodas. If his fridge would only hold 7 on each shelf, how many shelves would he fill up? Equation: x = (4.0 + 52.0)/7.0 Solution: 8 Table 1: A math word problem. The core idea is to leverage the immense capac- ity of neural networks to strengthen the process of equation generating. Compared to statistical machine learning-based methods (Kushman et al., 2014; Mitra and Baral, 2016; Roy and Roth, 2018; Zhou et al., 2015; Huang et al., 2016) and se- mantic parsing-based methods (Shi et al., 2015; Koncel-Kedziorski et al., 2015; Roy and Roth, 2015; Huang et al., 2017), these methods do not need hand-crafted features and achieve high per- formance on large datasets. However, they lack in capturing the specific MWPs features, which are an evidently vital component in solving MWP. More related work and feature-related information can be found in Zhang et al. (2018). Inspired by recent work on modeling local- …

Content truncated to 500 words