Contrastive Domain Adaptation for Question Answering using Limited Text Corpora

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9575–9593 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 9575 Contrastive Domain Adaptation for Question Answering using Limited Text Corpora Zhenrui Yue Technical University of Munich zhenrui.yue@tum.de Bernhard Kratzwald ETH AI Center bkratzwald@ethz.ch Stefan Feuerriegel ETH Zurich sfeuerriegel@ethz.ch Abstract Question generation has recently shown im- pressive results in customizing question an- swering (QA) systems to new domains. These approaches circumvent the need for manu- ally annotated training data from the new do- main and, instead, generate synthetic question- answer pairs that are used for training. How- ever, existing methods for question generation rely on large amounts of synthetically gen- erated datasets and costly computational re- sources, which render these techniques widely inaccessible when the text corpora is of lim- ited size. This is problematic as many niche domains rely on small text corpora, which nat- urally restricts the amount of synthetic data that can be generated. In this paper, we pro- pose a novel framework for domain adaptation called contrastive domain adaptation for QA (CAQA). Specifically, CAQA combines tech- niques from question generation and domain- invariant learning to answer out-of-domain questions in settings with limited text corpora. Here, we train a QA system on both source data and generated data from the target domain with a contrastive adaptation loss that is incor- porated in the training objective. By combin- ing techniques from question generation and domain-invariant learning, our model achieved considerable improvements compared to state- of-the-art baselines. 1 Introduction Question answering (QA) systems generate an- swers to questions over text. Formally, such sys- tems are nowadays trained end-to-end to predict answers conditional on an input question and a con- text paragraph (e.g., Seo et al., 2016; Chen et al., 2017a; Devlin et al., 2019). Therein, every QA sample is a 3-tuple consisting of a question, a con- text, and an answer. In this paper, we consider the subproblem of extractive QA, where the task is to extract answer spans from an unstructured context information for a given question as input. In extrac- Figure 1: Overview of a common framework for QA domain adaptation. A question generation model is used to generate synthetic target data, which can be used for training the QA model with source data. The resulting QA model can answer target questions upon deployment. tive QA, both question and context are represented by running text, while the answer is defined by a start position and an end position in the context. An existing challenge for extractive QA systems is the distributional change between training data (source domain) and test data (target domain). If there is such a distribution change, the performance on test data is likely to be impaired. In practice, this issue occurs due to the fact that users, for in- stance, formulate text in highly diverse language or use QA for previously unseen domains (Hazen et al., 2019; Miller et al., 2020). As a result, out-of- domain (OOD) samples occur that diverge from …

Content truncated to 500 words