Supervised and Unsupervised Transfer Learning for Question Answering

Proceedings of NAACL-HLT 2018, pages 1585–1594 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Supervised and Unsupervised Transfer Learning for Question Answering Yu-An Chung1 Hung-Yi Lee2 James Glass1 1MIT Computer Science and Artificial Intelligence Laboratory, Cambridge, MA, USA {andyyuan,glass}@mit.edu 2Department of Electrical Engineering, National Taiwan University, Taipei, Taiwan hungyilee@ntu.edu.tw Abstract Although transfer learning has been shown to be successful for tasks like object and speech recognition, its applicability to question answering (QA) has yet to be well-studied. In this paper, we con- duct extensive experiments to investigate the transferability of knowledge learned from a source QA dataset to a target dataset using two QA models. The per- formance of both models on a TOEFL lis- tening comprehension test (Tseng et al., 2016) and MCTest (Richardson et al., 2013) is significantly improved via a simple transfer learning technique from MovieQA (Tapaswi et al., 2016). In par- ticular, one of the models achieves the state-of-the-art on all target datasets; for the TOEFL listening comprehension test, it outperforms the previous best model by 7%. Finally, we show that transfer learn- ing is helpful even in unsupervised sce- narios when correct answers for target QA dataset examples are not available. 1 Introduction 1.1 Question Answering One of the most important characteristics of an in- telligent system is to understand stories like hu- mans do. A story is a sequence of sentences, and can be in the form of plain text (Trischler et al., 2017; Rajpurkar et al., 2016; Weston et al., 2016; Yang et al., 2015) or spoken content (Tseng et al., 2016), where the latter usually requires the spo- ken content to be first transcribed into text by au- tomatic speech recognition (ASR), and the model will subsequently process the ASR output. To evaluate the extent of the model’s understanding of the story, it is asked to answer questions about the story. Such a task is referred to as question answering (QA), and has been a long-standing yet challenging problem in natural language process- ing (NLP). Several QA scenarios and datasets have been in- troduced over the past few years. These scenarios differ from each other in various ways, including the length of the story, the format of the answer, and the size of the training set. In this work, we focus on context-aware multi-choice QA, where the answer to each question can be obtained by re- ferring to its accompanying story, and each ques- tion comes with a set of answer choices with only one correct answer. The answer choices are in the form of open, natural language sentences. To cor- rectly answer the question, the model is required to understand and reason about the relationship be- tween the sentences in the story. 1.2 Transfer Learning Transfer learning (Pan and Yang, 2010) is a vi- tal machine learning technique that aims to use the knowledge learned from one task and apply it to a different, but related, task in order to ei- ther reduce the necessary fine-tuning …

Content truncated to 500 words