Question Generation for Reading Comprehension Assessment by Modeling How and What to Ask

Findings of the Association for Computational Linguistics: ACL 2022, pages 2131 - 2146 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Question Generation for Reading Comprehension Assessment by Modeling How and What to Ask Bilal Ghanem1, Lauren Lutz Coleman2, Julia Rivard Dexter2, Spencer McIntosh von der Ohe1, Alona Fyshe1 1University of Alberta, Canada 2EyeRead, Canada {bghanem,vonderoh,alona}@ualberta.ca {lauren,julia}@eyeread.co Abstract Reading is integral to everyday life, and yet learning to read is a struggle for many young learners. During lessons, teachers can use com- prehension questions to increase engagement, test reading skills, and improve retention. His- torically such questions were written by skilled teachers, but recently language models have been used to generate comprehension questions. However, many existing Question Generation (QG) systems focus on generating literal ques- tions from the text, and have no way to control the type of the generated question. In this pa- per, we study QG for reading comprehension where inferential questions are critical and ex- tractive techniques cannot be used. We propose a two-step model (HTA-WTA) that takes ad- vantage of previous datasets, and can generate questions for a specific targeted comprehension skill. We propose a new reading comprehen- sion dataset that contains questions annotated with story-based reading comprehension skills (SBRCS), allowing for a more complete reader assessment. Across several experiments, our results show that HTA-WTA outperforms mul- tiple strong baselines on this new dataset. We show that the HTA-WTA model tests for strong SCRS by asking deep inferential questions. 1 Introduction Reading is an invaluable skill, and is core to com- municating in our digital age. Reading also sup- ports other forms of development; when children read, it sharpens their memory, and improves social skills (Halliday, 1973; Mason, 2017). Yet, statistics show that one out of five children in the U.S. face learning difficulties (Shaywitz, 2005), especially in reading (Cornoldi and Oakhill, 2013). The coro- navirus pandemic beginning in 2020 had a huge impact on the early reading skills of many children, and threatens to leave a lasting impact on a whole generation of young readers (Gupta and Jawanda, 2020). The pandemic forced many children to learn on- line, putting in sharp relief the need for effective online education platforms. In particular, reading games have become popular, and can help fill the gap when teachers cannot read in person with stu- dents. These platforms present students with short passages and associated comprehension questions. These questions are key to assessing a reader’s comprehension of a passage, and can also enhance learning (Chua et al., 2017). But, writing diverse and engaging comprehension questions is a non- trivial task. Teachers need to generate new comprehension questions whenever they incorporate new text into a curriculum. New text helps to keep material fresh and topical, and can allow teachers to customize lessons to the interests of a particular student co- hort. After finding such custom reading material, teachers must write new comprehension questions to evaluate several reading aspects of comprehen- sion (e.g. understanding complex words, recalling events, etc.). Thus, to improve …

Content truncated to 500 words