Reversing Gradients in Adversarial Domain Adaptation for Question Deduplication and Textual Entailment Tasks

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5545–5550 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5545 Reversing Gradients in Adversarial Domain Adaptation for Question Deduplication and Textual Entailment Tasks Anusha Kamath Carnegie Mellon University Pittsburgh, PA akamath1@andrew.cmu.edu Sparsh Gupta University of California San Diego San Diego, CA spg005@ucsd.edu Vitor Carvalho Intuit AI San Diego, CA vitor carvalho@intuit.com Abstract Adversarial domain adaptation has been re- cently introduced as an effective technique for textual matching tasks, such as question dedu- plication (Shah et al., 2018). Here we investi- gate the use of gradient reversal on adversar- ial domain adaptation to explicitly learn both shared and unshared (domain specific) repre- sentations between two textual domains. In doing so, gradient reversal learns features that explicitly compensate for domain mismatch, while still distilling domain specific knowl- edge that can improve target domain accuracy. We evaluate reversing gradients for adversarial adaptation on multiple domains, and demon- strate that it significantly outperforms other methods on question deduplication as well as on recognizing textual entailment (RTE) tasks, achieving up to 7% absolute boost in base model accuracy on some datasets. 1 Introduction Domain adaptation is a flexible machine learning approach that allows the transfer of category inde- pendent information between domains. Through domain adaptation we can leverage source task representations to bring the source and target dis- tributions closer in a learned joint feature space. In this paper we are focused only on semi-supervised domain adaptation — when knowledge from a large labeled dataset in a source domain can be somewhat transferred to help improve the same task on a target domain, which typically has a sig- nificantly smaller number of labels. In particu- lar, this paper focuses on domain adaptation for the detection of question duplicates in commu- nity question answering forums (Shah et al., 2018; Hoogeveen et al., 2015), as well as for RTE tasks (Dagan et al., 2005; Zhao et al., 2017). Generally speaking, the effectiveness of domain adaptation depends essentially on two factors: the similarity between source and target domains, and representation strategy to transfer the source do- main knowledge. Long et al. showed transfer- ring features across domains becomes increasingly difficult as domain discrepancy increases (Long et al., 2017), since the features learned by models gradually transition from general to highly domain specific as training progresses. Recent domain adaptation strategies attempt to counter this issue by making certain features invariant across source and target domains using distribution matching (Cao et al., 2018) or minimizing distance metrics between the representations (Sohn et al., 2019). The idea of generating domain invariant fea- tures was further enhanced by the use of adver- sarial learning methods. Recent work has advo- cated for tuning networks using a loss functions that reduce the mismatch between source and tar- get data distributions (Sankaranarayanan et al., 2018; Tzeng et al., 2017). Others have proposed a domain discriminator that maximizes the domain classification loss between source and target do- mains …

Content truncated to 500 words