Domain Adversarial Fine-Tuning as an Effective Regularizer

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3103–3112 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 3103 Domain Adversarial Fine-Tuning as an Effective Regularizer Giorgos Vernikos1, Katerina Margatina2, Alexandra Chronopoulou3, Ion Androutsopoulos4 1 School of ECE, National Technical University of Athens, Greece 2 Computer Science Department, University of Sheffield, UK 3 Center for Information and Language Processing, LMU Munich, Germany 4 Department of Informatics, Athens University of Economics and Business, Greece gvernikos@mail.ntua.gr, k.margatina@sheffield.ac.uk, achron@cis.lmu.de ion@aueb.gr Abstract In Natural Language Processing (NLP), pre- trained language models (LMs) that are trans- ferred to downstream tasks have been recently shown to achieve state-of-the-art results. How- ever, standard fine-tuning can degrade the general-domain representations captured dur- ing pretraining. To address this issue, we intro- duce a new regularization technique, AFTER; domain Adversarial Fine-Tuning as an Effec- tive Regularizer. Specifically, we complement the task-specific loss used during fine-tuning with an adversarial objective. This additional loss term is related to an adversarial classifier, that aims to discriminate between in-domain and out-of-domain text representations. In- domain refers to the labeled dataset of the task at hand while out-of-domain refers to unla- beled data from a different domain. Intuitively, the adversarial classifier acts as a regularizer which prevents the model from overfitting to the task-specific domain. Empirical results on various natural language understanding tasks show that AFTER leads to improved perfor- mance compared to standard fine-tuning. 1 Introduction Current research in NLP focuses on transferring knowledge from a language model (LM), pre- trained on large general-domain data, to a target task. The LM representations are transferred to the target task either as additional features of a task-specific model (Peters et al., 2018), or by fine- tuning (Howard and Ruder, 2018; Devlin et al., 2019; Yang et al., 2019). Standard fine-tuning in- volves initializing the target model with the pre- trained LM and training it with the target data. Fine-tuning, however, can lead to catastrophic forgetting (Goodfellow et al., 2013), if the pre- trained LM representations are adjusted to such an extent to the target task, that most generic knowl- edge, captured during pretraining, is in effect for- gotten (Howard and Ruder, 2018). A related prob- lem of fine-tuning is overfitting to the target task, that often occurs when only a small number of training examples is available (Dai and Le, 2015). Adversarial training is a method to increase robustness and regularize deep neural networks (Goodfellow et al., 2015; Miyato et al., 2017). It has been used for domain adaptation (Ganin et al., 2016) to train a model from scratch to produce rep- resentations that are invariant to different domains. Inspired by this approach, we propose a regular- ization technique for the fine-tuning process of a pretrained LM, that aims to optimize knowledge transfer to the target task and avoid overfitting. Our method, domain Adversarial Fine-Tuning as an Effective Regularizer (AFTER) extends standard fine-tuning by adding an adversarial objective to the task-specific loss. We leverage out-of-domain unlabeled data (i.e. from a different domain than …

Content truncated to 500 words