EPT-X: An Expression-Pointer Transformer model that generates eXplanations for numbers

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 4442 - 4458 May 22-27, 2022 c⃝2022 Association for Computational Linguistics EPT-X: An Expression-Pointer Transformer model that generates eXplanations for numbers Bugeun Kim Kyung Seo Ki Sangkyu Rhim Gahgene Gweon Department of Intelligence and Information, Seoul National University Seoul, Republic of Korea {cd4209, kskee88, sk.rhim, ggweon}@snu.ac.kr Abstract In this paper, we propose a neural model EPT- X (Expression-Pointer Transformer with Ex- planations), which utilizes natural language ex- planations to solve an algebraic word problem. To enhance the explainability of the encoding process of a neural model, EPT-X adopts the concepts of plausibility and faithfulness which are drawn from math word problem solving strategies by humans. A plausible explanation is one that includes contextual information for the numbers and variables that appear in a given math word problem. A faithful expla- nation is one that accurately represents the reasoning process behind the model’s solution equation. The EPT-X model yields an average baseline performance of 69.59% on our PEN dataset and produces explanations with quality that is comparable to human output. The con- tribution of this work is two-fold. (1) EPT-X model: An explainable neural model that sets a baseline for algebraic word problem solving task, in terms of model’s correctness, plausi- bility, and faithfulness. (2) New dataset: We release a novel dataset PEN (Problems with Explanations for Numbers), which expands the existing datasets by attaching explanations to each number/variable. 1 Introduction Algebraic word problem solving is a challenging task for understanding natural language. As shown in Table 1, a model needs to interpret a word problem into a solution equation to solve the problem. Recent neural approaches have employed encoder-decoder architecture to tackle this task and achieved remarkable answer correctness (Huang et al., 2018; Chiang and Chen, 2019; Amini et al., 2019; Kim et al., 2020; Ki et al., 2020): ranging from 65% to 84% depending on datasets. So, as the model delivers a plausible answer, researchers have a firm belief that an encoder component of a neural model can comprehend the problem correctly. However, this belief has less been verified due to Q. Tom has 12 coins in quarters and nickels. Their value is $2.20. How many coins of each type does he have? Solution x + y = 12, 0.25x + 0.05y = 2.20 Equation (∴ x = 8, y = 4) Explanation 12 = the total number of coins x2.20 = the total value of coins = the number of quarters y = the number of nickels Table 1: An example algebraic word problem the opaqueness of an encoder. Thus, this paper aims to design an architecture that reveals how it understands a given word problem. Our novel model, Expression-Pointer Trans- former with Explanations (EPT-X), is inspired by some pedagogical studies about human strate- gies on understanding an algebraic word problem (Conway and Polya, 1985; Jitendra et al., 2007; Montague, 2008; Jitendra and Star, 2012). In classrooms, teachers ask students to …

Content truncated to 500 words