Let’s Ask Again: Refine Network for Automatic Question Generation

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3314–3323, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 3314 Let’s Ask Again: Refine Network for Automatic Question Generation Preksha Nema†‡∗ Akash Kumar Mohankumar†∗ Mitesh M. Khapra†‡ Balaji Vasan Srinivasan• Balaraman Ravindran†‡ †IIT Madras, India •Adobe Research ‡ Robert Bosch Center for Data Science and Artificial Intelligence, IIT Madras {preksha,miteshk,ravi}@cse.iitm.ac.in makashkumar99@gmail.com {balsrini}@adobe.com Abstract In this work, we focus on the task of Au- tomatic Question Generation (AQG) where given a passage and an answer the task is to generate the corresponding question. It is de- sired that the generated question should be (i) grammatically correct (ii) answerable from the passage and (iii) specific to the given an- swer. An analysis of existing AQG mod- els shows that they produce questions which do not adhere to one or more of the above- mentioned qualities. In particular, the gener- ated questions look like an incomplete draft of the desired question with a clear scope for refinement. To alleviate this shortcoming, we propose a method which tries to mimic the human process of generating questions by first creating an initial draft and then refin- ing it. More specifically, we propose Re- fine Network (RefNet) which contains two de- coders. The second decoder uses a dual at- tention network which pays attention to both (i) the original passage and (ii) the question (initial draft) generated by the first decoder. In effect, it refines the question generated by the first decoder, thereby making it more cor- rect and complete. We evaluate RefNet on three datasets, viz., SQuAD, HOTPOT-QA, and DROP, and show that it outperforms exist- ing state-of-the-art methods by 7-16% on all of these datasets. Lastly, we show that we can improve the quality of the second decoder on specific metrics, such as, fluency and answer- ability by explicitly rewarding revisions that improve on the corresponding metric during training. The code has been made publicly available 1. 1 Introduction Over the past few years, there has been a growing interest in Automatic Question Generation (AQG) work.∗* The first two authors have contributed equally to this 1https://github.com/PrekshaNema25/RefNet-QG Passage 1: Liberated by Napoleon’s army in 1806, Warsaw was made the capital of the newly created Duchy of Warsaw. Generated Questions Baseline What was the capital of the newly duchy of Warsaw? RefNet Who liberated Warsaw in 1806? Reward-RefNet Whose army liberated Warsaw in 1806? Passage 2: To fix carbon dioxide into sugar molecules in the process of photosynthesis, chloroplasts use an en- zyme called rubisco Generated Questions Baseline What does chloroplasts use? RefNet What does chloroplasts use to fix carbon dioxide into sugar molecules? Reward-RefNet What do chloroplasts use to fix carbon dioxide into sugar molecules? Table 1: Samples of generated questions from Base- line, RefNet and Reward-RefNet model on the SQuAD dataset. Answers are shown in blue . from text - the task of generating a question from a …

Content truncated to 500 words