Domain Adaptation for Question Answering via Question Classification

Proceedings of the 29th International Conference on Computational Linguistics, pages 1776–1790 October 12–17, 2022. 1776 Domain Adaptation for Question Answering via Question Classification Zhenrui Yue and Huimin Zeng and Ziyi Kou and Lanyu Shang and Dong Wang School of Information Sciences University of Illinois Urbana-Champaign {zhenrui3, huiminz3, ziyikou2, lshang3, dwang24}@illinois.edu Abstract Question answering (QA) has demonstrated im- pressive progress in answering questions from customized domains. Nevertheless, domain adaptation remains one of the most elusive challenges for QA systems, especially when QA systems are trained in a source domain but deployed in a different target domain. In this work, we investigate the potential bene- fits of question classification for QA domain adaptation. We propose a novel framework: Question Classification for Question Answer- ing (QC4QA). Specifically, a question classifier is adopted to assign question classes to both the source and target data. Then, we perform joint training in a self-supervised fashion via pseudo- labeling. For optimization, inter-domain dis- crepancy between the source and target domain is reduced via maximum mean discrepancy (MMD) distance. We additionally minimize intra-class discrepancy among QA samples of the same question class for fine-grained adap- tation performance. To the best of our knowl- edge, this is the first work in QA domain adap- tation to leverage question classification with self-supervised adaptation. We demonstrate the effectiveness of the proposed QC4QA with consistent improvements against the state-of- the-art baselines on multiple datasets. 1 Introduction Question Answering (QA) or Reading Compre- hension (RC) refers to the task of extracting an- swers from given context paragraphs based on in- put questions. QA systems predict the start and end positions of possible answer spans in given context documents upon input questions. In re- cent studies, QA systems have achieved significant improvements with transformer models and large- scale datasets (Rajpurkar et al., 2016; Devlin et al., 2019; Yue et al., 2022a). Once deployed, QA systems often experience performance deterioration upon user-generated questions. Such performance drops can be traced Figure 1: Overview for QA domain adaptation. A QA model is trained with labeled source data and unlabeled target data. The resulting QA system is deployed to answer target questions. back to domain shifts in two input elements: (1) User-generated questions are syntactically more diverse and thus, different from the training QA pairs; (2) The context domain of test-time input (target domain) can oftentimes diverge from the training corpora (source domain), e.g., from news snippets to biomedical articles (Hazen et al., 2019; Fisch et al., 2019; Miller et al., 2020). To alleviate the performance issue in QA do- main adaptation, several approaches have been pro- posed to reduce the discrepancy between the source and target domains. Integrating labeled target QA pairs in training can effectively improve the QA system in answering out-of-domain questions (Ka- math et al., 2020; Shakeri et al., 2020; Yue et al., 2021, 2022b), where the target data can be human- annotated QA pairs or synthetic data using ques- tion generation methods. When only unlabeled questions are available (see Figure 1), another pos- …

Content truncated to 500 words