Cycle-Consistent Adversarial Autoencoders for Unsupervised Text Style Transfer

Proceedings of the 28th International Conference on Computational Linguistics, pages 2213–2223 Barcelona, Spain (Online), December 8-13, 2020 2213 Cycle-Consistent Adversarial Autoencoders for Unsupervised Text Style Transfer Yufang Huang1 Wentao Zhu2 Deyi Xiong3 Yiye Zhang4 Changjian Hu1 Feiyu Xu1 1Lenovo Research AI Lab, 2Kwai Inc., 3Tianjin University, 4Cornell University {yfhuang1992new,wentaozhu91}@gmail.com, dyxiong@tju.edu.cn yiz2014@med.cornell.edu, {hucj1,fxu}@lenovo.com Abstract Unsupervised text style transfer is full of challenges due to the lack of parallel data and difficulties in content preservation. In this paper, we propose a novel neural approach to unsupervised text style transfer which we refer to as Cycle-consistent Adversarial autoEncoders (CAE) trained from non-parallel data. CAE consists of three essential components: (1) LSTM autoencoders that encode a text in one style into its latent representation and decode an encoded representation into its original text or a transferred representation into a style-transferred text, (2) adversarial style transfer networks that use an adversarially trained generator to transform a latent representation in one style into a representation in another style, and (3) a cycle-consistent constraint that enhances the capacity of the adversarial style transfer networks in content preservation. The entire CAE with these three components can be trained end-to-end. Extensive experiments and in-depth analyses on two widely-used public datasets consistently validate the effectiveness of proposed CAE in both style transfer and content preservation against several strong baselines in terms of four automatic evaluation metrics and human evaluation. 1 Introduction Unsupervised text style transfer is to rewrite a text in one style into a text in another style while the content of the text remains the same as much as possible without using any parallel data. Style trans- fer can be utilized in many tasks such as personalization in dialogue systems (Oraby et al., 2018; Colombo et al., 2019), sentiment and word decipherment (Shen et al., 2017), offensive language transla- tion (Nogueira dos Santos et al., 2018), and data augmentation (Perez and Wang, 2017; Mikołajczyk and Grochowski, 2018; Zhu et al., 2020), etc. However, there are a variety of challenges to text style transfer in practice. First, we do not have large-scale style-to-style parallel data to train a text style transfer model in a supervised way. Second, even with non-parallel corpora, the inherent discrete structure of text sequences aggravates the difficulty of learning desirable continuous representations for style transfer (Huang et al., 2020; Zhao et al., 2018; Bowman et al., 2016; Hjelm et al., 2018). Third, it is difficult to preserve the content of a text when its style is transferred. To obtain good content preservation for text style transfer, various disentanglement approaches (Shen et al., 2017; Hu et al., 2017; Fu et al., 2018; Sudhakar et al., 2019) are proposed to separate the content and style of a text in the latent space. However, content-style disentanglement is not easily achievable as content and style typically interact with each other in texts in subtle ways (Lample et al., 2019). In order to solve the issues above, we propose a cycle-consistent adversarial autoencoders (CAE) for unsupervised text style transfer. In …

Content truncated to 500 words