ProblemSolver at SemEval-2019 Task 10: Sequence-to-Sequence Learning and Expression Trees

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1292–1296 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1292 ProblemSolver at SemEval-2019 Task 10: Sequence-to-Sequence Learning and Expression Trees Xuefeng Luo, Alina Baranova, Jonas Biegert Linguistics Department, University of Tuebingen, Germany firstname.lastname@student.uni-tuebingen.de Abstract This paper describes our participation in SemEval-2019 shared task “Math Question Answering”, where the aim is to create a pro- gram that could solve the Math SAT ques- tions automatically as accurately as possible. We went with a dual-pronged approach, build- ing a Sequence-to-Sequence Neural Network pre-trained with augmented data that could an- swer all categories of questions and a Tree sys- tem, which can only answer a certain type of questions. The systems did not perform well on the entire test data given in the task, but did decently on the questions they were actu- ally capable of answering. The Sequence-to- Sequence Neural Network model managed to get slightly better than our baseline of guess- ing “A” for every question, while the Tree sys- tem additionally improved the results. 1 Introduction The data set for the task (Hopkins et al., 2019) includes questions used in the Math SAT. There are three broad categories of questions: closed- vocabulary and open-vocabulary algebra ques- tions, and geometry questions. All types of questions consist in large part of natural lan- guage. Closed-vocabulary algebra questions typ- ically contain math equations and many math- specific words, while open-vocabulary algebra questions include more everyday vocabulary and quantities which are expressed in letters, numbers or a combination of both. Geometry questions are usually provided with diagrams the analysis of which is necessary for solving the problems. Most questions of all categories are multiple- choice questions with five possible options, but some questions have a numeric answer. We present two systems to tackle these math problems. One of them, a sequence-to-sequence LSTM model pre-trained with augmented data, is applied to all three types of questions, while the other, a system based on expression trees produces answers exclusively for open-vocabulary algebra questions. 2 Related Work In their work, Roy and Roth (2015) introduced bi- nary expression trees that represent and solve math word problems. To choose the best expression tree out of all possible trees, the authors employed two classifiers: a relevance classifier, which de- termined if the quantity should be included into the expression tree, and a Lowest Common Ances- tor (LCA) classifier, which output the most proba- ble mathematical operation for a pair of quantities. Both classifiers were trained on gold annotations. Subsequently, two other systems were devel- oped based on Roy and Roth (2015). One of the systems belongs to the same authors and uses the concept of Unit Dependency Graphs (UDGs) to capture the information between units of the quantities (Roy and Roth, 2017). UDGs are then united with expression trees, allowing the infor- mation about dependencies between units improve the math problem solver. Another system (Wang et al., 2018) suggests a method to improve Roy and …

Content truncated to 500 words