Low Resource Style Transfer via Domain Adaptive Meta Learning

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3014 - 3026 July 10-15, 2022 ©2022 Association for Computational Linguistics Low Resource Style Transfer via Domain Adaptive Meta Learning Xiangyang Li1,∗, Xiang Long2, ∗, Yu Xia1, Sujian Li1, † 1Key Laboratory of Computational Linguistics, Peking University, MOE, China 2Beijing University of Posts and Telecommunications, Beijing, China {xiangyangli, yuxia, lisujian}@pku.edu.cn xianglong@bupt.edu.cn Abstract Text style transfer (TST) without parallel data has achieved some practical success. How- ever, most of the existing unsupervised text style transfer methods suffer from (i) requiring massive amounts of non-parallel data to guide transferring different text styles. (ii) colossal performance degradation when fine-tuning the model in new domains. In this work, we pro- pose DAML-ATM (Domain Adaptive Meta- Learning with Adversarial Transfer Model), which consists of two parts: DAML and ATM. DAML is a domain adaptive meta-learning approach to learn general knowledge in mul- tiple heterogeneous source domains, capable of adapting to new unseen domains with a small amount of data. Moreover, we pro- pose a new unsupervised TST approach Ad- versarial Transfer Model (ATM), composed of a sequence-to-sequence pre-trained language model and uses adversarial style training for better content preservation and style transfer. Results on multi-domain datasets demonstrate that our approach generalizes well on unseen low-resource domains, achieving state-of-the- art results against ten strong baselines. 1 Introduction Text style transfer (TST) aims to change the style of the input text and keep its content unchanged, which has been applied successfully to text formal- ization (Jain et al., 2019) , text rewriting (Nikolov and Hahnloser, 2018) , personalized dialogue gen- eration (Niu and Bansal, 2018) and other stylized text generation tasks (Gao et al., 2019; Cao et al., 2020; Syed et al., 2020). Text style transfer has been explored as a sequence-to-sequence learning task using parallel datasets (Jhamtani et al., 2017; Wang et al., 2020b; Pryzant et al., 2020). However, parallel datasets are difficult to obtain due to expensive manual annota- tion. The recent surge of deep generative methods ∗ Equal contribution. † Corresponding author. (Hu et al., 2017a; Zhao et al., 2017; Li et al., 2018) has spurred progress in text style transfer without parallel data. However, these methods typically re- quire large amounts of non-parallel data and do not perform well in low-resource domain scenarios. One typical method is to resort to massive data from different domains, which has been studied as an effective solution to address the above data insufficiency issue (Glorot et al., 2011; Wang et al., 2017; Li et al., 2021b). However, directly leverag- ing large amounts of data from other domains for the TST task is problematic due to the differences in data distribution over different domains, as dif- ferent domains usually use their domain-specific lexica (Li et al., 2019a). For instance, fine-tuning a TST model trained on a high-resource movie- related domain to a low-resource restaurant-related domain can get us unreasonable sentences like "the food is dramatic." The sentiment word …

Content truncated to 500 words