Mapping probability word problems to executable representations

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3627–3640 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 3627 Mapping probability word problems to executable representations Simon ˇSuster⋆⋄ Pieter Fivez♠ Pietro Totis♣ Angelika Kimmig♣ Jesse Davis♣ Luc De Raedt♣ Walter Daelemans♠ ⋆University of Melbourne ♠University of Antwerp ♣KU Leuven ⋄simon.suster@unimelb.edu.au Abstract While solving math word problems automati- cally has received considerable attention in the NLP community, few works have addressed probability word problems specifically. In this paper, we employ and analyse various neural models for answering such word problems. In a two-step approach, the problem text is first mapped to a formal representation in a declar- ative language using a sequence-to-sequence model, and then the resulting representation is executed using a probabilistic programming system to provide the answer. Our best per- forming model incorporates general-domain contextualised word representations that were finetuned using transfer learning on another in-domain dataset. We also apply end-to-end models to this task, which bring out the im- portance of the two-step approach in obtaining correct solutions to probability problems. 1 Introduction Solving math word problems automatically is an active area of research in natural language process- ing. It poses interesting challenges in extracting relevant entities and quantities from a concise tex- tual narrative, and reasoning about the relation- ships between them to answer the question posed in the text. While several approaches for different areas of mathematics have been proposed in the past, the majority have focused on arithmetic and algebraic problems where the mathematical repre- sentation is specified as a simple equation system that can be used with a standard solver (Hosseini et al., 2014; Roy and Roth, 2015; Kushman et al., 2014; Koncel-Kedziorski et al., 2015; Upadhyay and Chang, 2015; Huang et al., 2016; Wang et al., 2017; Ling et al., 2017; Amini et al., 2019; Miao et al., 2020). This paper focuses on a class of word problems that have not received much atten- tion: those about probability that may be found, for example, in introductory textbooks for discrete mathematics. Problem text: A complete cycle of a traffic light takes 80 seconds. During each cycle, the light is green for 40 seconds, amber for 10 seconds, and red for 30 seconds. At a randomly chosen time, what is the probability that the light will not be red? Representation: group(cycle). size(cycle, 80). property(property, [amber, green, red]). given(exactly(10, cycle, amber)). given(exactly(30, cycle, red)). given(exactly(40, cycle, green)). take(cycle, light, 1). probability(none(light, red)). Answer: 5/8 Table 1: An example from the NLP4PLP dataset (Dries et al., 2017): input text, intermediate representation, and correct answer. Table 1 provides an example problem. The over- all task, to which we adhere in this work, is to obtain the answer from the text, with the formal representation as an intermediate step. Generally, the representation contains one or more multisets of objects with certain properties, one or more ac- tions that create new multisets, and a question that imposes a constraint on the result of the actions and …

Content truncated to 500 words