Multiple Text Style Transfer by using Word-level Conditional Generative Adversarial Network with Two-Phase Training

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3579–3584, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 3579 Multiple Text Style Transfer by using Word-level Conditional Generative Adversarial Network with Two-Phase Training Chih-Te Lai1, Yi-Te Hong1,2, Hong-You Chen1, Chi-Jen Lu1, Shou-De Lin2 1Academina Sinica, Taipei, Taiwan 2National Taiwan University, Taipei, Taiwan 1{cloudylai, ted0504, cbbjames, cjlu}@iis.sinica.edu.tw 2sdlin@ntu.edu.tw Abstract The objective of non-parallel text style trans- fer is to alter specific attributes (e.g. senti- ment, mood, tense, politeness, etc) of a given text while preserving unrelated content. Ad- versarial training is a popular method to en- sure the transferred sentences have the desired target styles. However, previous works of- ten suffer from content leaking problem. In this paper, we propose a new adversarial train- ing model with a word-level conditional ar- chitecture and a two-phase training procedure. By using a style-related condition architec- ture before generating a word, our model is able to maintain style-unrelated words while changing the others. By separating the train- ing procedure into reconstruction and transfer phases, our model is able to balance the re- construction and adversarial losses. We test our model on polarity sentiment transfer and multiple-attribute transfer tasks. The empirical results show that our model achieves compara- ble evaluation scores in both transfer accuracy and fluency but significantly outperforms other state-of-the-art models in content compatibil- ity on three real-world datasets. 1 Introduction Text style transfer is a challenging problem in nat- ural language generation, whose objective is to al- ter specific attributes (e.g. sentiment, mood, tense, voice, politeness, etc (Hu et al., 2017; Shen et al., 2017; Sennrich et al., 2016; Logeswaran et al., 2018; Prabhumoye et al., 2018)) of a given text while preserving its remaining attributes and con- tents. This task has potential applications such as paraphrasing, summarizing articles, author ob- fuscation (Reddy and Knight, 2016), poems/lyrics rewriting, and scenario-adaptive machine transla- tion (Michel and Neubig, 2018). One major challenge for text style transfer is that parallel data across different text attributes is difficult to collect and label. Without parallel data, supervised deep-learning models are not applica- ble and the transfer rules among styles unclear. Unlike image style transfer, another main chal- lenge for text style transfer is the difficulty to iden- tify and disentangle neural feature representations for texts. In previous models (Shen et al., 2017; Hu et al., 2017) the core idea for non-parallel text style transfer, is to training an auto-encoder with addi- tional adversarial loss (Goodfellow et al., 2014), (or a VAE with a classifier), for the discriminator, (or classifier), to guide the decoder generate sen- tences to have a specific target style. Despite previous success in attribute- conditioned text generation, several research questions remain, regarding to previous models, including: 1) previous modeling limitations to transformations between only a few attributes. 2) the issue of trade-off among text fluency, content preservation, and the accurate transfer with the desired attributes and 3) …

Content truncated to 500 words