Iterative GNN-based Decoder for Question Generation

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2573–2582 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 2573 Iterative GNN-based Decoder for Question Generation Zichu Fei, Qi Zhang, Yaqian Zhou Shanghai Key Laboratory of Intelligent Information Processing School of Computer Science, Fudan University, Shanghai, China {zcfei19, qz, yqzhou}@fudan.edu.cn Abstract Natural question generation (QG) aims to gen- erate questions from a passage, and gener- ated questions are answered from the passage. Most models with state-of-the-art performance model the previously generated text at each de- coding step. However, (1) they ignore the rich structure information that is hidden in the pre- viously generated text. (2) they ignore the im- pact of copied words on the passage. We per- ceive that information in previously generated words serves as auxiliary information in subse- quent generation. To address these problems, we design the Iterative Graph Network-based Decoder (IGND) to model the previous gener- ation using a Graph Neural Network at each decoding step. Moreover, our graph model captures dependency relations in the passage that boost the generation. Experimental results demonstrate that our model outperforms the state-of-the-art models with sentence-level QG tasks on SQuAD and MARCO datasets. 1 Introduction Automatic Question Generation (QG) is the task of generating question-answer pairs from a declara- tive sentence, QG has many useful applications: (1) it improves the question answering task (Chen et al., 2017) by providing more training data (Tang et al., 2017; Yuan et al., 2017); (2) it generates practice exercises and assessments for educational purposes (Heilman and Smith, 2010); and (3) it helps dialog systems to kick-start and continue a conversation with human users (Mostafazadeh et al., 2016). In this study, we focus on sentence-level QG tasks. Conventional QG methods (Mostow and Chen, 2009; Heilman and Smith, 2010; Dhole and Man- ning, 2020) rely on heuristic rules or hand-crafted templates as it suffers a significant lack of ques- tion, sticking to a few simple and reliable syntactic transformation patterns. Recently, neural-based ap- proaches to QG have achieved remarkable success, by applying large-scale reading comprehension Figure 1: An example (lower-cased) of using the struc- ture information that is hidden in previously gener- ated words information and the impact of copied word, where the answer words are in blue and the copied words are in purple. The model can copy the right word develop with a high certainty (0.97 score). datasets and employing the encoder-decoder frame- work. Most of the existing works are based on the sequence-to-sequence (Seq2Seq) network, incorpo- rating the attention mechanism and copy mode, ap- plied by (Zhou et al., 2018). Intuitively, connecting an answer to a passage lies at the heart of this task. (Song et al., 2018) leveraged multi-perspective matching methods and (Sun et al., 2018) proposed a position-aware model to put more emphasis on answer-surrounded context words. (Zhao et al., 2018) aggregated the paragraph-level context to provide sufficient information for question genera- tion. (Chen et al., 2020; Liu et al., 2019) employed the Graph2Seq architecture to capture the …

Content truncated to 500 words