ComSearch: Equation Searching with Combinatorial Strategy for Solving Math Word Problems with Weak Supervision

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2549–2562 May 2-6, 2023 ©2023 Association for Computational Linguistics ComSearch: Equation Searching with Combinatorial Strategy for Solving Math Word Problems with Weak Supervision Qianying Liu1, Wenyu Guan2, Jianhao Shen3, Fei Cheng1 and Sadao Kurohashi1 1 Graduate School of Informatics, Kyoto University 2 Xiaomi AI Lab 3 Peking University ying@nlp.ist.i.kyoto-u.ac.jp; gwy1995@live.com; jhshen@pku.edu.cn; {feicheng,kuro}@nlp.ist.i.kyoto-u.ac.jp; Abstract Previous studies have introduced a weakly- supervised paradigm for solving math word problems requiring only the answer value an- notation. While these methods search for cor- rect value equation candidates as pseudo la- bels, they search among a narrow sub-space of the enormous equation space. To address this problem, we propose a novel search algo- rithm with combinatorial strategy ComSearch, which can compress the search space by exclud- ing mathematically equivalent equations. The compression allows the searching algorithm to enumerate all possible equations and obtain high-quality data. We investigate the noise in the pseudo labels that hold wrong mathematical logic, which we refer to as the false-matching problem, and propose a ranking model to de- noise the pseudo labels. Our approach holds a flexible framework to utilize two existing su- pervised math word problem solvers to train pseudo labels, and both achieve state-of-the-art performance in the weak supervision task. 1 1 Introduction Solving math word problems (MWPs) is the task of extracting a mathematical solution from problems written in natural language. Based on a sequence- to-sequence (seq2seq) framework that takes in the text descriptions of the MWPs and predicts the an- swer equation (Wang et al., 2017), task-specialized encoder and decoder architectures (Wang et al., 2018b, 2019; Xie and Sun, 2019; Liu et al., 2019; Guan et al., 2019; Zhang et al., 2020b,a; Shen and Jin, 2020), data augmentation and normaliza- tion (Wang et al., 2018a; Liu et al., 2020; Shen et al., 2022b), and pretrained models (Tan et al., 2021; Liang et al., 2021; Shen et al., 2021, 2022a) have been conducted on full supervision setting of the task. These settings require equation ex- 1Our code and data is available at https://github.com/ yiyunya/ComSearch Figure 1: Example of MWP solving system under full supervision and weak supervision. pression annotation, which is expensive and time- consuming. Recently Hong et al. (2021) (LBF) and Chatter- jee et al. (2021) (WARM) addressed this problem and proposed the weak supervision setting, where only the answer value annotation is given for su- pervision. Such a setting forms pseudo question- candidate equation pairs, which hold the correct answer value for training with the complexity of O(n2n) for n variables enormous possible equa- tion space. Computational efficiently extracting such pairs becomes the major challenge since it is computationally impossible to traverse all pos- sible equations, especially when the example has more variables (e.g., 88,473,600 for 6 variables). As we show in Figure 1, previous studies sample a limited set of equations via random walk (Hong et al., 2021) or beam searching (Chatterjee et al., 2021). However, the algorithms …

Content truncated to 500 words