A Reinforcement Learning Framework for Natural Question Generation using Bi-discriminators

Proceedings of the 27th International Conference on Computational Linguistics, pages 1763–1774 Santa Fe, New Mexico, USA, August 20-26, 2018. 1763 A Reinforcement Learning Framework for Natural Question Generation using Bi-discriminators Zhihao Fan1, Zhongyu Wei1∗, Siyuan Wang1, Yang Liu2, Xuanjing Huang3 1School of Data Science, Fudan University, China 2Liulishuo Company 3School of Computer Science, Fudan University, China {14300180043,zywei,14302010062}@fudan.edu.cn, yang.liu@liulishuo.com, xjhuang@fudan.edu.cn Abstract Visual Question Generation (VQG) aims to ask natural questions about an image automatically. Existing research focuses on training model to fit the annotated data set that makes it indifferent from other language generation tasks. We argue that natural questions need to have two specific textual attributes from the perspectives of content and linguistic respectively, namely, natural and human-written. Inspired by the setting of discriminator in adversarial learning, we propose two discriminators, one for each attribute, to enhance the training. We then use the reinforcement learning framework to incorporate scores from the two discriminators as the reward to guide the training of the question generator. Experimental results on a benchmark VQG dataset show the effectiveness and robustness of our model compared to some state-of-the-art models in terms of both automatic and human evaluation metrics. 1 Introduction Recent years see the popularity of multi-modal research on vision and language. Visual caption gen- eration (VCG) (Xu et al., 2015; Vinyals et al., 2015) and visual question answering (VQA) (Antol et al., 2015) attract increasing attention from research communities. VCG aims to generate descriptions for a given image with the goal of scene understanding, while VQA asks visual questions and requires an answer to it. Research for these two tasks are fueled by several manually generated corpora (Lin et al., 2014; Zhu et al., 2016). Different from generating a statement (descriptions or answers) about an image, visual question gen- eration (VQG) (Mostafazadeh et al., 2016) is tasked with generating a natural question which can potentially engage a human in starting a conversation when shown an image. Under this guidance, Mostafazadeh et al. (2016) collect natural questions for images via a crowd-sourcing platform and con- struct the first dataset for VQG. They also explore some neural network-based models for natural question generation. Those models are trained to better fit the VQG dataset that makes them indifferent from other language generation models and hard to identify the progress in naturalness for generated ones. Some human generated questions for VQG and questions for VQA are shown in Figure 1, we name questions for VQA descriptive questions and those for VQG natural ones. As we can see, VQA questions are much simpler and can be easily answered using information from the source image directly. In contrast, VQG questions are more complex and answers are not trivial. We therefore argue that the specialty in terms of content needs to be considered for natural question generation. In this paper, We formulate the task of visual natural question generation as language generation task with specific attributes in terms of content and linguistics, i.e. natural and human-written. Recently, adversarial learning approaches (Goodfellow …

Content truncated to 500 words