Improving Unsupervised Question Answering via Summarization-Informed Question Generation

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4134–4148 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 4134 Improving Unsupervised Question Answering via Summarization-Informed Question Generation Chenyang Lyu† Lifeng Shang‡ Yvette Graham¶ Jennifer Foster† Xin Jiang‡ Qun Liu‡ † School of Computing, Dublin City University, Dublin, Ireland ‡ Huawei Noah’s Ark Lab, Hong Kong, China ¶ School of Computer Science and Statistics, Trinity College Dublin, Dublin, Ireland chenyang.lyu2@mail.dcu.ie, ygraham@tcd.ie, jennifer.foster@dcu.ie {shang.lifeng, jiang.xin, qun.liu}@huawei.com Abstract Question Generation (QG) is the task of gen- erating a plausible question for a given <pas- sage, answer> pair. Template-based QG uses linguistically-informed heuristics to trans- form declarative sentences into interrogatives, whereas supervised QG uses existing Question Answering (QA) datasets to train a system to generate a question given a passage and an an- swer. A disadvantage of the heuristic approach is that the generated questions are heavily tied to their declarative counterparts. A disadvan- tage of the supervised approach is that they are heavily tied to the domain/language of the QA dataset used as training data. In order to overcome these shortcomings, we propose an unsupervised QG method which uses ques- tions generated heuristically from summaries as a source of training data for a QG sys- tem. We make use of freely available news summary data, transforming declarative sum- mary sentences into appropriate questions us- ing heuristics informed by dependency pars- ing, named entity recognition and semantic role labeling. The resulting questions are then combined with the original news articles to train an end-to-end neural QG model. We ex- trinsically evaluate our approach using unsu- pervised QA: our QG model is used to gen- erate synthetic QA pairs for training a QA model. Experimental results show that, trained with only 20k English Wikipedia-based syn- thetic QA pairs, the QA model substantially outperforms previous unsupervised models on three in-domain datasets (SQuAD1.1, Natural Questions, TriviaQA) and three out-of-domain datasets (NewsQA, BioASQ, DuoRC), demon- strating the transferability of the approach. 1 Introduction The aim of Question Generation (QG) is the pro- duction of meaningful questions given a set of input passages and corresponding answers, a task with many applications including dialogue systems as Figure 1: Example questions generated via heuristics informed by semantic role labeling of summary sen- tences using different candidate answer spans well as education (Graesser et al., 2005). Addition- ally, QG can be applied to Question Answering (QA) for the purpose of data augmentation (Puri et al., 2020) where labeled <passage, answer, ques- tion> triples are combined with synthetic <passage, answer, question> triples produced by a QG system to train a QA system, and unsupervised QA (Lewis et al., 2019), in which only the QG system output is used to train the QA system. Early work on QG focused on template or rule- based approaches, employing syntactic knowledge to manipulate constituents in declarative sentences to form interrogatives (Heilman and Smith, 2009, 2010). Although template-based methods are ca- pable of generating linguistically correct questions, the resulting questions often lack variety and …

Content truncated to 500 words