Reinforcement Learning Based Text Style Transfer without Parallel Training Corpus

Proceedings of NAACL-HLT 2019, pages 3168–3180 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 3168 Reinforcement Learning Based Text Style Transfer without Parallel Training Corpus Hongyu Gong* Suma Bhat* Lingfei Wu† Jinjun Xiong † Wen-mei Hwu* *University of Illinois at Urbana-Champaign, USA †T. J. Watson Research Center, IBM *{hgong6, spbhat2, w-hwu}@illinois.edu †{wuli, jinjun}@us.ibm.com Abstract Text style transfer rephrases a text from a source style (e.g., informal) to a target style (e.g., formal) while keeping its original mean- ing. Despite the success existing works have achieved using a parallel corpus for the two styles, transferring text style has proven sig- nificantly more challenging when there is no parallel training corpus. In this paper, we ad- dress this challenge by using a reinforcement- learning-based generator-evaluator architec- ture. Our generator employs an attention- based encoder-decoder to transfer a sentence from the source style to the target style. Our evaluator is an adversarially trained style dis- criminator with semantic and syntactic con- straints that score the generated sentence for style, meaning preservation, and fluency. Ex- perimental results on two different style trans- fer tasks (sentiment transfer and formality transfer) show that our model outperforms state-of-the-art approaches. Furthermore, we perform a manual evaluation that demonstrates the effectiveness of the proposed method using subjective metrics of generated text quality. 1 Introduction Text style transfer is the task of rewriting a piece of text to a particular style while retaining the meaning of the original text. It is a challenging task of natural language generation and is at the heart of many recent NLP applications, such as personalized responses in dialogue system (Zhou et al., 2017), formalized texts (Rao and Tetreault, 2018), cyberspace purification by rewriting offen- sive texts (Niu and Bansal, 2018; Santos et al., 2018), and poetry generation (Yang et al., 2018). Recent works on supervised style transfer with a parallel corpus have demonstrated considerable success (Jhamtani et al., 2017b; Rao and Tetreault, 2018). However, a parallel corpus may not al- ways be available for a transfer task. This has prompted studies on style transfer without paral- lel corpora. These hinge on the common idea of separating the content from the style of the text (Shen et al., 2017; Fu et al., 2018; Santos et al., 2018). This line of research first encodes the context via a style-independent representation, and then transfers sentences by combining the en- coded content with style information. In addition, an appropriate training loss is chosen to change the style while preserving the content. However, these approaches are limited by their use of loss functions that must be differentiable with respect to the model parameters, since they rely on gra- dient descent to update the parameters. Further- more, since focusing only on semantic and style metrics in style transfer, they ignore other impor- tant aspects of quality in text generation, such as language fluency. In this paper, we propose a system trained us- ing reinforcement-learning (RL) that performs text style transfer without accessing to a parallel …

Content truncated to 500 words