HAWP: a Dataset for Hindi Arithmetic Word Problem Solving

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 3479–3490 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 3479 HAWP: a Dataset for Hindi Arithmetic Word Problem Solving Harshita Sharma, Pruthwik Mishra, Dipti Misra Sharma LTRC, IIIT-Hyderabad {harshita.sharma, pruthwik.mishra}@research.iiit.ac.in, dipti@iiit.ac.in Abstract Word Problem Solving remains a challenging and interesting task in NLP. A lot of research has been carried out to solve different genres of word problems with various complexity levels in recent years. However, most of the publicly available datasets and work has been carried out for English. Recently there has been a surge in this area of word problem solving in Chinese with the creation of large benchmark datastes. Apart from these two languages, labeled benchmark datasets for low resource languages are very scarce. This is the first attempt to address this issue for any Indian Language, especially Hindi. In this paper, we present HAWP (Hindi Arithmetic Word Problems), a dataset consisting of 2336 arithmetic word problems in Hindi. We also developed baseline systems for solving these word problems. We also propose a new evaluation technique for word problem solvers taking equation equivalence into account. Keywords: word problem solving, lexical diversity, low resource, biLSTM 1. Introduction Math Word Problems (MWPs), mainly taught in pri- mary and high schools, are considered to be an inte- gral part of Math curriculum across the globe. MWPs are seen as brain teasers because they expect the solver to understand a hypothetical situation involving quan- tities using some statements that must be used to an- swer a question about that situation. For a machine to correctly model this type of information, form relation- ships among the quantities and generate the answer is a very complex task. Most often MWPs are designed as real world narratives and act as a mental stimuli for the students. Recent works have shown that solving elementary-level word problems itself still poses a sig- nificant challenge for the NLP community. Although difficult, word problem solving has been approached with a number of innovative and intelligent perspec- tives: semantic parsing, template matching, explain- able solvers etc. As a result of these different approaches, a large num- ber of datasets of various levels of complexity have been developed in languages like Chinese and English. However, no substantial dataset exists for Indian Lan- guages. To address this, we have created a dataset for Hindi Word Problem Solving. To make the data more diverse we adopted a 2-pronged strategy to construct a diverse and challenging dataset of 2336 MWPs. Some of these MWPs were collected from Hindi textbooks and worksheets and also from Hindi-medium educators while most of them were augmented using translation. Through our paper, we make the following contribu- tions: • We tackled the problem of lack of Word problem solving dataset and solvers in Hindi language. • We designed a good quality, diverse and challeng- ing, publicly available1 dataset of 2336 MWPs an- 1https://github.com/hellomasaya/hawp notated with equations, number of operations and …

Content truncated to 500 words