AiFu at SemEval-2019 Task 10: A Symbolic and Sub-symbolic Integrated System for SAT Math Question Answering

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 900–906 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 900 AiFu at SemEval-2019 Task 10: A Symbolic and Sub-symbolic Integrated System for SAT Math Question Answering Keyu Ding1,2,6, Yifan Liu1,2, Yi Zhou3,5, Binbin Deng1,2, Chaoyang Peng1,2, Dinglong Xue1,2, Qinzhuo Wu3, Qi Zhang3 and Enhong Chen5 1iFLYTEK Research 2State Key Laboratory of Cognitive Intelligence, iFLYTEK, P.R. China 3Shanghai Research Center for Brain Science and Brain-Inspired Intelligence/ZhangJiang Lab 4School of Computer Science, Shanghai Key Laboratory of Intelligent Information Processing, Fudan University 5School of Natural and Computational Sciences, Massey University 6University of Science and Technology of China {kyding,yfliu7,bbdeng,zypeng,dlxue}@iflytek.com yzhou@bsbii.cn {qzwu17,qz}@fudan.edu.cn cheneh@ustc.edu.cn Abstract AiFu has won the first place in the SemEval- 2019 Task [10] - ”Math Question Answer- ing”(Hopkins et al.) competition. This paper is to describe how it works technically and to report and analyze some essential experimen- tal results. 1 Introduction Recently, math question answering has attracted a lot of attention in the AI community, both in academia and in industry (Matsuzaki et al., 2017) (Wang et al., 2017) (Huang et al., 2016) (Wang et al., 2018) (Hosseini et al., 2014)(Huang et al., 2018a) (Huang et al., 2018b) (Kushman et al., 2014) (Liang et al., 2017) (Mitra and Baral, 2016) (Zhou et al., 2015) (Roy and Roth, 2017)(Hopkins et al., 2017). On one side, it raises a difficult yet workable challenge for the current development of AI research. In order to tackle this challenge, one has to integrate and advance many subareas in AI including knowledge representation and rea- soning, machine learning, natural language under- standing and image understanding. On the other side, math question answering itself has important commercial value in the AI+Education industry. Against this backdrop, SemEval-2019 orga- nizes a competition on math question answering, namely Task 10 (Hopkins et al.). In this task, an opportunity is provided for Math Question- Answering systems to test themselves on a bench- mark that consists of many math questions col- lected from the US Math Scholastic Achievement Test (SAT). We have implemented a prototype system, called AiFu, to tackle this challenge, and it has won the first place at the end. AiFu is an in- tegrated system that combines the state-of-the-art approaches from many important subareas in AI, and more importantly, it also develops and justifies some new ideas and techniques. This paper is to describe how AiFu works tech- nically and to report and analyze some essential experimental results. In the next section, we go through the technical details of AiFu that consists of many essential components including represen- tation, reasoning and natural language understand- ing. In Section 3, we report our experimental re- sults and shed new insights on why AiFu works in some cases but not in others. Finally, we conclude this paper and point out some future directions. 2 Method Figure 1 depicts the overall architecture of AiFu. Given a mathematical question, a translator is used to convert it into its internal representation, which …

Content truncated to 500 words