Domain Adaptive Text Style Transfer

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3304–3313, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 3304 Domain Adaptive Text Style Transfer Dianqi Li1, Yizhe Zhang2, Zhe Gan2, Yu Cheng2, Chris Brockett2, Ming-Ting Sun1, Bill Dolan2 1University of Washington 2Microsoft Corporation {dianqili,mts}@uw.edu {Yizhe.Zhang,Zhe.Gan,Yu.Cheng,Chris.Brockett,billdol}@microsoft.com Abstract Text style transfer without parallel data has achieved some practical success. However, in the scenario where less data is available, these methods may yield poor performance. In this paper, we examine domain adaptation for text style transfer to leverage massively available data from other domains. These data may demonstrate domain shift, which impedes the benefits of utilizing such data for training. To address this challenge, we propose simple yet effective domain adaptive text style trans- fer models, enabling domain-adaptive infor- mation exchange. The proposed models pre- sumably learn from the source domain to: (i) distinguish stylized information and generic content information; (ii) maximally preserve content information; and (iii) adaptively trans- fer the styles in a domain-aware manner. We evaluate the proposed models on two style transfer tasks (sentiment and formality) over multiple target domains where only limited non-parallel data is available. Extensive ex- periments demonstrate the effectiveness of the proposed model compared to the baselines. 1 Introduction Text style transfer, which aims to edit an input sentence with the desired style while preserving style-irrelevant content, has received increasing at- tention in recent years. It has been applied suc- cessfully to stylized image captioning (Gan et al., 2017), personalized conversational response gen- eration (Zhang et al., 2018a), formalized writ- ing (Rao and Tetreault, 2018), offensive to non- offensive language transfer (dos Santos et al., 2018), and other stylized text generation tasks (Akama et al., 2017; Zhang et al., 2019). Text style transfer has been explored as a sequence-to-sequence learning task using parallel datasets (Jhamtani et al., 2017). However, par- allel datasets are often not available, and hand- annotating sentences in different styles is expen- sive. The recent surge of deep generative mod- els (Kingma and Welling, 2013; Goodfellow et al., 2014) has spurred progress in text style trans- fer without parallel data by learning disentangle- ment (Hu et al., 2017; Shen et al., 2017; Fu et al., 2018; Li et al., 2018; Prabhumoye et al., 2018). These methods typically require massive amounts of data (Subramanian et al., 2018), and may per- form poorly in limited data scenarios. A natural solution to the data-scarcity issue is to resort to massive data from other domains. However, directly leveraging abundant data from other domains is problematic due to the discrep- ancies in data distribution on different domains. Different domains generally manifest themselves in domain-specific lexica. For example, senti- ment adjectives such as “delicious”, “tasty”, and “disgusting” in restaurant reviews might be out of place in movie reviews, where the sentiment words such as “imaginative”, “hilarious”, and “dramatic” are more typical. Domain shift (Gret- ton et al., 2009) is thus apt to …

Content truncated to 500 words