Asking Questions Like Educational Experts: Automatically Generating Question-Answer Pairs on Real-World Examination Data

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2583–2593 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 2583 Asking Questions Like Educational Experts: Automatically Generating Question-Answer Pairs on Real-World Examination Data Fanyi Qu, Xin Jia, Yunfang Wu∗ Key Laboratory of Computational Linguistics (Peking University), Ministry of Education Department of Computer Science and Technology, Peking University {fanyiqu, jemmryx, wuyf}@pku.edu.cn Abstract Generating high quality question-answer pairs is a hard but meaningful task. Although pre- vious works have achieved great results on answer-aware question generation, it is diffi- cult to apply them into practical application in the education field. This paper for the first time addresses the question-answer pair generation task on the real-world examination data, and proposes a new unified framework on RACE. To capture the important information of the input passage we first automatically generate (rather than extracting) keyphrases, thus this task is reduced to keyphrase-question-answer triplet joint generation. Accordingly, we propose a multi-agent communication model to generate and optimize the question and keyphrases iteratively, and then apply the gen- erated question and keyphrases to guide the generation of answers. To establish a solid benchmark, we build our model on the strong generative pre-training model. Experimental results show that our model makes great break- throughs in the question-answer pair genera- tion task. Moreover, we make a comprehen- sive analysis on our model, suggesting new di- rections for this challenging task. 1 Introduction Question-answer pair generation (QAG) is to do question generation (QG) and answer generation (AG) simultaneously only with a given passage. The generated question-answer (Q-A) pairs can be effectively applied in numbers of tasks such as knowledge management (Wagner and Bolloju, 2005), FAQ document generation (Krishna and Iyyer, 2019), and data enhancement for reading comprehension tasks (Tang et al., 2017; Liu et al., 2020a,b). Particularly, high-quality Q-A pairs can facilitate the instructing process and benefit on cre- ating educational materials for reading practice and assessment (Heilman, 2011; Jia et al., 2020). ∗Corresponding author Figure 1: A brief overview of our proposed framework. Recently, much work has devoted to QG, while the QAG task is less addressed. The existing ap- proaches on this task can be roughly grouped into two categories: the joint learning method to treat QG and answer extraction as dual tasks (Collobert et al., 2011; Firat et al., 2016); the pipeline strategy that considers answer extraction and QG as two sequential processes (Cui et al., 2021). Although some progress has been made, the QAG methods still face several challenges. Most of the existing techniques are trained and tested on the Web-extracted corpora like SQuAD (Rajpurkar et al., 2016), MARCO (Nguyen et al., 2016) and NewsQA (Trischler et al., 2017). Considering the biased and unnatural language sources of datasets, employing these techniques in educational field is difficult. Moreover, most of the previous works regard the answer text as a continuous span in the passage and directly obtain the answer through an extractive method, which may not meet the de- mands of real world data. …

Content truncated to 500 words