Self-Training for Jointly Learning to Ask and Answer Questions

Proceedings of NAACL-HLT 2018, pages 629–640 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Self-Training for Jointly Learning to Ask and Answer Questions Mrinmaya Sachan Eric P. Xing School of Computer Science Carnegie Mellon University {mrinmays, epxing}@cs.cmu.edu Abstract Building curious machines that can answer as well as ask questions is an important challenge for AI. The two tasks of question answering and question generation are usually tackled separately in the NLP literature. At the same time, both require significant amounts of su- pervised data which is hard to obtain in many domains. To alleviate these issues, we pro- pose a self-training method for jointly learning to ask as well as answer questions, leveraging unlabeled text along with labeled question an- swer pairs for learning. We evaluate our ap- proach on four benchmark datasets: SQUAD, MS MARCO, WikiQA and TrecQA, and show significant improvements over a number of es- tablished baselines on both question answer- ing and question generation tasks. We also achieved new state-of-the-art results on two competitive answer sentence selection tasks: WikiQA and TrecQA. 1 Introduction Question Answering (QA) is a well-studied prob- lem in NLP which focuses on answering questions using some structured or unstructured sources of knowledge. Alongside question answering, there has also been some work on generating ques- tions (QG) (Heilman, 2011; Du et al., 2017; Tang et al., 2017) which focuses on generating ques- tions based on given sources of knowledge. QA and QG are closely related1 tasks. However, NLP literature views the two as entirely separate tasks. In this paper, we explore this relationship between the two tasks by jointly learning to gen- erate as well as answer questions. An improved ability to generate as well as answer questions will help us build curious machines that can interact with humans in a better manner. Joint modeling of 1We can think of QA and QG as inverse of each other. QA and QG is useful as the two can be used in con- junction to generate novel questions from free text and then answers for the generated questions. We use this idea to perform self-training (Nigam and Ghani, 2000) and leverage free text to augment the training of QA and QG models. QA and QG models are typically trained on question answer pairs which are expensive to ob- tain in many domains. However, it is cheaper to obtain large quantities of free text. Our self- training procedure leverages unlabeled text to boost the quality of our QA and QG models. This is achieved by a careful data augmentation proce- dure which uses pre-trained QA and QG models to generate additional labeled question answer pairs. This additional data is then used to retrain our QA and QG models and the procedure is repeated. This addition of synthetic labeled data needs to be performed carefully. During self-training, typically the most confident samples are added to the training set (Zhu, 2005) in each iteration. We use the performance of our QA and QG …

Content truncated to 500 words