PIE-QG: Paraphrased Information Extraction for Unsupervised Question Generation from Small Corpora

Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL), pages 350 - 359 December 7-8, 2022 ©2022 Association for Computational Linguistics PIE-QG: Paraphrased Information Extraction for Unsupervised Question Generation from Small Corpora Dinesh Nagumothu, Bahadorreza Ofoghi, Guangyan Huang, Peter W. Eklund School of Information Technology, Deakin University, 221 Burwood Hwy, Burwood 3125 Victoria, Australia {dnagumot,b.ofoghi,guangyan.huang,peter.eklund}@deakin.edu.au Abstract Supervised Question Answering systems (QA systems) rely on domain-specific human- labeled data for training. Unsupervised QA systems generate their own question-answer training pairs, typically using secondary knowl- edge sources to achieve this outcome. Our ap- proach (called PIE-QG) uses Open Information Extraction (OpenIE) to generate synthetic train- ing questions from paraphrased passages and uses the question-answer pairs as training data for a language model for a state-of-the-art QA system based on BERT. Triples in the form of <subject, predicate, object> are extracted from each passage, and questions are formed with subjects (or objects) and predicates while ob- jects (or subjects) are considered as answers. Experimenting on five extractive QA datasets demonstrates that our technique achieves on- par performance with existing state-of-the-art QA systems with the benefit of being trained on an order of magnitude fewer documents and without any recourse to external reference data sources. 1 Introduction Question Answering systems (QA systems) pro- vide answers to input questions posed in natural lan- guage. Answering questions from unstructured text can be performed using Machine Reading Com- prehension (MRC). Given a passage, several sen- tences or a paragraph, and a question posed, the QA system produces the best suitable answer. Ex- tractive Question Answering systems (EQA sys- tems) are a subset of QA systems and involve an MRC task where the predicted answer is a span of words from the passage. With pre-trained lan- guage models (Radford et al., 2018), EQA systems achieve excellent results, surpassing even human performance. Pre-trained language models, such as BERT (Devlin et al., 2019) and GPT (Radford et al.), can be fine-tuned to perform downstream tasks such as QA. However, huge amounts of data are required to train these models for specific do- mains, making the task labor-intensive, in terms of the effort required to assemble suitable domain- specific training data. A single training instance for an EQA system dataset requires a question, a passage, and an an- swer. Domain-relevant documents can be collected with advanced information retrieval tools, and pas- sages are formed by splitting documents into sev- eral related sentences or a paragraph. However, generating the question and answer pairs, that pro- vide the training set for the QA system from a given passage, is considered the most difficult challenge, an approach known as unsupervised QA (Cui et al., 2004). Existing unsupervised QA system techniques such as (Lewis et al., 2019) and (Lyu et al., 2021) use an out-of-domain dataset for question gen- eration, namely, they require additional training sources beyond what can be provided by the target corpus and a pre-trained generic model. On the other hand, rule-based QA system methods, those constrained to generate question-answer …

Content truncated to 500 words