DoCoGen: Domain Counterfactual Generation for Low Resource Domain Adaptation

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7727 - 7746 May 22-27, 2022 c⃝2022 Association for Computational Linguistics DoCoGen: Domain Counterfactual Generation for Low Resource Domain Adaptation Nitay Calderon∗and Eyal Ben-David∗and Amir Feder and Roi Reichart Technion - Israel Institute of Technology {nitay@campus.|eyalbd12@campus.|feder@campus.|roiri@}technion.ac.il Abstract Natural language processing (NLP) algorithms have become very successful, but they still struggle when applied to out-of-distribution examples. In this paper we propose a con- trollable generation approach in order to deal with this domain adaptation (DA) challenge. Given an input text example, our DoCoGen algorithm generates a domain-counterfactual textual example (D-CON) – that is similar to the original in all aspects, including the task la- bel, but its domain is changed to a desired one. Importantly, DoCoGen is trained using only unlabeled examples from multiple domains – no NLP task labels or parallel pairs of textual examples and their domain-counterfactuals are required. We show that DoCoGen can gener- ate coherent counterfactuals consisting of mul- tiple sentences. We use the D-CONs generated by DoCoGen to augment a sentiment classifier and a multi-label intent classifier in 20 and 78 DA setups, respectively, where source-domain labeled data is scarce. Our model outperforms strong baselines and improves the accuracy of a state-of-the-art unsupervised DA algorithm.1 1 Introduction Natural Language Processing (NLP) algorithms are constantly improving and reaching significant milestones (Devlin et al., 2019; Raffel et al., 2020; Brown et al., 2020). However, such algorithms rely on the availability of sufficient labeled data and the assumption that the training and test sets are drawn from the same underlying distribution. Unfortunately, these assumptions do not hold in many cases due to the costly and labor-intensive data labeling process and since text may originate from many different domains. As generalization in low resource regimes and beyond the training distribution are still fundamental NLP challenges, ∗Both authors equally contributed to this work. 1Our code and data are available at https://github. com/nitaytech/DoCoGen. NLP algorithms significantly degrade when applied to such scenarios. Domain adaptation (DA) is an established field of research in NLP (Roark and Bacchiani, 2003; Daumé III and Marcu, 2006; Reichart and Rap- poport, 2007) that attempts to explicitly address generalization beyond the training distribution (§2). DA algorithms are trained on annotated data from source domains to be effectively applied in various target domains. Indeed, DA algorithms have been developed for multiple NLP tasks throughout the last two decades (Blitzer et al., 2006, 2007; Glorot et al., 2011; Rush et al., 2012; Ziser and Reichart, 2017, 2018a,b; Han and Eisenstein, 2019). A natural alternative to costly human annotation would be to automatically generate labeled exam- ples for model training. Doing so may expose the model to additional training examples and better represent the data distribution within and outside the annotated source domains. Unfortunately, gen- erating labeled textual data is challenging (Feng et al., 2021), especially when the available labeled data is scarce. Indeed, labeled data generation has hardly been applied to DA (§2). …

Content truncated to 500 words