Multi-Task Learning with Language Modeling for Question Generation

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3394–3399, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 3394 Multi-Task Learning with Language Modeling for Question Generation Wenjie Zhou, Minghua Zhang, Yunfang Wu∗ Key Laboratory of Computational Linguistics, Ministry of Education School of Electronics Engineering and Computer Science, Peking University, Beijing, China {wjzhou013,zhangmh,wuyf}@pku.edu.cn Abstract This paper explores the task of answer-aware questions generation. Based on the attention- based pointer generator model, we propose to incorporate an auxiliary task of language modeling to help question generation in a hi- erarchical multi-task learning structure. Our joint-learning model enables the encoder to learn a better representation of the input se- quence, which will guide the decoder to gen- erate more coherent and fluent questions. On both SQuAD and MARCO datasets, our multi- task learning model boosts the performance, achieving state-of-the-art results. Moreover, human evaluation further proves the high qual- ity of our generated questions. 1 Introduction Question generation (QG) receives increasing in- terests in recent years due to its benefits to several real applications: (1) QG can aid in the develop- ment of annotated questions to boost the question answering systems (Duan et al., 2017; Tang et al., 2017); (2) QG enables the dialogue systems to ask questions which make it more proactive (Shum et al., 2018; Colby, 1975); (3) QG can help to gen- erate questions for reading comprehension texts in the education field. In this paper, we focus on answer-aware QG. Giving a sentence and an an- swer span as input, we want to generate a question whose response is the answer. Previous work on QG was mainly tackled by two approaches: the rule-based approach and neural-based approach. The neural-based ap- proach receives a booming development due to the release of large-scale reading comprehension datasets like SQuAD (Rajpurkar et al., 2016) and MARCO (Nguyen et al., 2016). Most of the neu- ral approaches on QG employ the encoder-decoder ∗Corresponding author. framework, which incorporate attention mecha- nism to pay more attention to the informative part and copy mode to copy some tokens from the input text (Du et al., 2017; Zhou et al., 2017; Song et al., 2018; Subramanian et al., 2018; Zhao et al., 2018; Sun et al., 2018). To make better use of answer information, Song et al. (2018) leverage multi- perspective matching, and Sun et al. (2018) pro- pose a position-aware model that aims at putting more emphasis on the answer-surrounded context words. Zhao et al. (2018) aggregate paragraph- level information to help QG. Another line of work is to deal with question answering and question generation as dual tasks (Tang et al., 2017; Duan et al., 2017). Some other works try to generate questions from a text without answers as input (Subramanian et al., 2018; Du and Cardie, 2017). Although some progress has been made, there is still much room for improvement for QG. Multi-task learning is an effective way to …

Content truncated to 500 words