Multi-source Meta Transfer for Low Resource Multiple-Choice Question Answering

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7331–7341 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 7331 Multi-source Meta Transfer for Low Resource Multiple-Choice Question Answering Ming Yan1, Hao Zhang1,2, Di Jin3, Joey Tianyi Zhou1,∗ 1Institute of High Performance Computing, A*STAR, Singapore 2SCSE, Nanyang Technological University, Singapore 3CSAIL, Massachusetts Institute of Technology, USA mingy@ihpc.a-star.edu.sg, zhang hao@ihpc.a-star.edu.sg jindi15@mit.edu, joey zhou@ihpc.a-star.edu.sg Abstract Multiple-choice question answering (MCQA) is one of the most challenging tasks in ma- chine reading comprehension since it requires more advanced reading comprehension skills such as logical reasoning, summarization, and arithmetic operations. Unfortunately, most ex- isting MCQA datasets are small in size, which increases the difficulty of model learning and generalization. To address this challenge, we propose a multi-source meta transfer (MMT) for low-resource MCQA. In this framework, we first extend meta learning by incorporat- ing multiple training sources to learn a gen- eralized feature representation across domains. To bridge the distribution gap between train- ing sources and the target, we further introduce the meta transfer that can be integrated into the multi-source meta training. More impor- tantly, the proposed MMT is independent of backbone language models. Extensive exper- iments demonstrate the superiority of MMT over state-of-the-arts, and continuous improve- ments can be achieved on different backbone networks on both supervised and unsupervised domain adaptation settings. 1 Introduction Recently, there has been a growing interest in mak- ing machines to understand human languages, and a great progress has been made in machine reading comprehension (MRC). There are two main types of MRC task: 1) extractive/abstractive question an- swering (QA) such as SQuAD (Rajpurkar et al., 2018) and DROP (Dua et al., 2019); 2) multiple- choice QA (MCQA) such as MultiRC (Khashabi et al., 2018) and DREAM (Sun et al., 2019a). Dif- ferent from extractive/abstractive QA whose an- swers are usually limited to the text spans exist in the passage, the answers of MCQA may not ap- pear in the text passage and may involve complex ∗Corresponding author. Meta Learning Multi-source Meta Transfer 1 3 2 Target Meta model MMT model Task in source 2 Task in source 3 Task in source 4 Task in source 1 Source representation MMT representation MML MTL Figure 1: Comparison of meta learning and multi- source meta transfer learning (MMT). “MTL” denotes meta transfer learning, and “MML” denotes multi- source meta learning. language inference. Thus, MCQA usually requires more advanced reading comprehension abilities, in- cluding arithmetic operation, summarization, logic reasoning and commonsense reasoning (Richard- son et al., 2013; Sun et al., 2019a), and etc. In ad- dition, the size of most existing MCQA datasets is much smaller than that of the extractive/abstractive QA datasets. For instance, all the span-based QA datasets, except CQ (Bao et al., 2016), contain more than 100k samples. In contrast, the data size of most existing MCQA datasets are far less than 100k (see Table 1), and the smallest one only con- tains 660 samples. The above two major challenges make MCQA much …

Content truncated to 500 words