KHANQ: A Dataset for Generating Deep Questions in Education

Proceedings of the 29th International Conference on Computational Linguistics, pages 5925–5938 October 12–17, 2022. 5925 KHANQ: A Dataset for Generating Deep Questions in Education Huanli Gong1 Liangming Pan2 Hengchang Hu2 1The Ohio State University, Columbus, OH, USA 2School of Computing, National University of Singapore, Singapore gong.545@osu.edu {liangmingpan, hengchanghu}@u.nus.edu Abstract Designing in-depth educational questions is a time-consuming and cognitively demanding task. Therefore, it is intriguing to study how to build Question Generation (QG) models to automate the question creation process. How- ever, existing QG datasets are not suitable for educational question generation because the questions are not real questions asked by hu- mans during learning and can be solved by simply searching for information. To bridge this gap, we present KHANQ, a chal- lenging dataset for educational question gen- eration, containing 1,034 high-quality learner- generated questions seeking an in-depth under- standing of the taught online courses in Khan Academy. Each data sample is carefully para- phrased and annotated as a triple of 1) Con- text: an independent paragraph on which the question is based; 2) Prompt: a text prompt for the question (e.g., the learner’s background knowledge); 3) Question: a deep question based on Context and coherent with Prompt. By conducting a human evaluation on the as- pects of appropriateness, coverage, coherence, and complexity, we show that state-of-the-art QG models which perform well on shallow question generation datasets have difficulty in generating useful educational questions. This makes KHANQ a challenging testbed for edu- cational question generation for further inves- tigation. 1 Introduction Question asking has long been considered a fun- damental cognitive process in learning. An ideal learner should be an active, self-motivated, creative, inquisitive person who asks deep questions and searches for answers to thought-provoking ques- tions, usually in the form of Why, Why-not, How, What-if, etc (Otero and Graesser, 2001). For exam- ple, Figure 1 shows a question raised by a learner after learning “allosteric regulation and feedback The learner 4 years ago Allosteric regulation confuses me a lot. I don't really get it even after I read the article on Khan Academy . Can anyone explain it to me briefly? Reply Comment (8 votes) l:::,,. Upvote V Downvote J:] Flag more v 00 The instructor 4 years ago I'll try an analogy - let me know if this helps. Imagine that an enzyme is like tiny sculpture made from a wire twisted into a very complicated, but somewhat loose structure. The substrate is another much smaller sculpture that fits into a gap in the first sculpture - let's say it fits perfectly. Now think of hanging a weight off another part of the sculpture - the whole structure shifts a bit under the strain and now the substrate sculpture doesn't fit! In this situation the weight would be analogous to an allosteric inhibitor. You could also imagine a similar scenario, but with the substrate fitting poorly until you added a weight - in this case the weight would be analogous to an allosteric activator. 2 comments (39 votes) …

Content truncated to 500 words