Semi-Supervised QA with Generative Domain-Adaptive Nets

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1040–1050 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1096 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1040–1050 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1096 Semi-Supervised QA with Generative Domain-Adaptive Nets Zhilin Yang Junjie Hu Ruslan Salakhutdinov William W. Cohen School of Computer Science Carnegie Mellon University {zhiliny,junjieh,rsalakhu,wcohen}@cs.cmu.edu Abstract We study the problem of semi-supervised question answering—-utilizing unlabeled text to boost the performance of ques- tion answering models. We propose a novel training framework, the Generative Domain-Adaptive Nets. In this framework, we train a generative model to gener- ate questions based on the unlabeled text, and combine model-generated questions with human-generated questions for train- ing question answering models. We de- velop novel domain adaptation algorithms, based on reinforcement learning, to alle- viate the discrepancy between the model- generated data distribution and the human- generated data distribution. Experiments show that our proposed framework obtains substantial improvement from unlabeled text. 1 Introduction Recently, various neural network models were proposed and successfully applied to the tasks of questions answering (QA) and/or reading com- prehension (Xiong et al., 2016; Dhingra et al., 2016; Yang et al., 2017). While achieving state- of-the-art performance, these models rely on a large amount of labeled data. However, it is extremely difficult to collect large-scale question answering datasets. Historically, many of the question answering datasets have only thousands of question answering pairs, such as WebQues- tions (Berant et al., 2013), MCTest (Richardson et al., 2013), WikiQA (Yang et al., 2015), and TREC-QA (Voorhees and Tice, 2000). Although larger question answering datasets with hundreds of thousands of question-answer pairs have been collected, including SQuAD (Rajpurkar et al., 2016), MSMARCO (Nguyen et al., 2016), and NewsQA (Trischler et al., 2016a), the data collec- tion process is expensive and time-consuming in practice. This hinders real-world applications for domain-specific question answering. Compared to obtaining labeled question answer pairs, it is trivial to obtain unlabeled text data. In this work, we study the following problem of semi-supervised question answering: is it possi- ble to leverage unlabeled text to boost the perfor- mance of question answering models, especially when only a small amount of labeled data is avail- able? The problem is challenging because con- ventional manifold-based semi-supervised learn- ing algorithms (Zhu and Ghahramani, 2002; Yang et al., 2016a) cannot be straightforwardly applied. Moreover, since the main foci of most question answering tasks are extraction rather than genera- tion, it is also not sensible to use unlabeled text to improve language modeling as in machine transla- tion (Gulcehre et al., 2015). To better leverage the unlabeled text, we pro- pose a novel neural framework called Genera- tive Domain-Adaptive Nets (GDANs). The start- ing point of our framework is to use linguistic tags to extract possible answer chunks in the un- labeled text, and then train a generative model to generate questions given the answer …

Content truncated to 500 words