Automatically Solving Number Word Problems by Semantic Parsing and Reasoning

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1132–1142, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Automatically Solving Number Word Problems by Semantic Parsing and Reasoning Shuming Shi1, Yuehui Wang2*, Chin-Yew Lin1, Xiaojiang Liu1 and Yong Rui1 1 Microsoft Research {shumings, cyl, xiaojl, yongrui}@microsoft.com 2 University of Science and Technology of China wyh9346@mail.ustc.edu.cn Abstract This paper presents a semantic parsing and reasoning approach to automatically solving math word problems. A new meaning representation language is de- signed to bridge natural language text and math expressions. A CFG parser is imple- mented based on 9,600 semi-automati- cally created grammar rules. We conduct experiments on a test set of over 1,500 number word problems (i.e., verbally ex- pressed number problems) and yield 95.4% precision and 60.2% recall. 1 Introduction Computers, since their creation, have exceeded human beings in (speed and accuracy of) mathe- matical calculation. However, it is still a big chal- lenge nowadays to design algorithms to automat- ically solve even primary-school-level math word problems (i.e., math problems described in natural language). Efforts to automatically solve math word prob- lems date back to the 1960s (Bobrow, 1964a, b). Previous work on this topic falls into two catego- ries: symbolic approaches and statistical learning methods. In symbolic approaches (Bobrow, 1964a, b; Charniak, 1968; Bakman, 2007; Liguda & Pfeiffer, 2012), math problem sentences are transformed to certain structures by pattern matching or verb categorization. Equations are then derived from the structures. Statistical learn- ing methods are employed in two recent papers (Kushman et al., 2014; Hosseini et al., 2014). Most (if not all) previous symbolic approaches suffer from two major shortcomings. First, natural language (NL) sentences are processed by simply applying pattern matching and/or transformation rules in an ad-hoc manner (refer to the related work section for more details). Second, surpris- ingly, they seldom report evaluation results about the effectiveness of the methods (except for some examples for demonstration purposes). For the small percentage of work with evaluation results available, it is unclear whether the patterns and rules are specially designed for specific sentences in a test set. Figure 1: Number word problem examples In this paper, we present a computer system called SigmaDolphin which automatically solves math word problems by semantic parsing and rea- soning. We design a meaning representation lan- guage called DOL (abbreviation of dolphin lan- guage) as the structured semantic representation of NL text. A semantic parser is implemented to transform math problem text into DOL trees. A reasoning module is included to derive math ex- pressions from DOL trees and to calculate final answers. Our approach falls into the symbolic cat- egory, but makes improvements over previous symbolic methods in the following ways, ______________________________________ * Work done while this author was an intern at Microsoft Research 1). One number is 16 more than another. If the smaller number is subtracted from 2/3 of the larger, the result is 1/4 of the sum of the two numbers. Find the numbers. 2). …

Content truncated to 500 words