Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5997–6007 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5997 Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation Ning Dai, Jianze Liang, Xipeng Qiu∗, Xuanjing Huang Shanghai Key Laboratory of Intelligent Information Processing, Fudan University School of Computer Science, Fudan University 825 Zhangheng Road, Shanghai, China {ndai16,jzliang18,xpqiu,xjhuang}@fudan.edu.cn Abstract Disentangling the content and style in the la- tent space is prevalent in unpaired text style transfer. However, two major issues exist in most of the current neural models. 1) It is difficult to completely strip the style informa- tion from the semantics for a sentence. 2) The recurrent neural network (RNN) based en- coder and decoder, mediated by the latent rep- resentation, cannot well deal with the issue of the long-term dependency, resulting in poor preservation of non-stylistic semantic content. In this paper, we propose the Style Trans- former, which makes no assumption about the latent representation of source sentence and equips the power of attention mechanism in Transformer to achieve better style transfer and better content preservation. Source code will be available on Github1. 1 Introduction Text style transfer is the task of changing the stylistic properties (e.g., sentiment) of the text while retaining the style-independent content within the context. Since the definition of the text style is vague, it is difficult to construct paired sen- tences with the same content and differing styles. Therefore, the studies of text style transfer focus on the unpaired transfer. Recently, neural networks have become the dominant methods in text style transfer. Most of the previous methods (Hu et al., 2017; Shen et al., 2017; Fu et al., 2018; Carlson et al., 2017; Zhang et al., 2018b,a; Prabhumoye et al., 2018; Jin et al., 2019; Melnyk et al., 2017; dos Santos et al., 2018) formulate the style transfer problem into the “encoder-decoder” framework. The en- coder maps the text into a style-independent latent ∗Corresponding author 1https://github.com/fastnlp/fastNLP representation (vector representation), and the de- coder generates a new text with the same content but a different style from the disentangled latent representation plus a style variable. These methods focus on how to disentangle the content and style in the latent space. The la- tent representation needs better preserve the mean- ing of the text while reducing its stylistic proper- ties. Due to lacking paired sentence, an adversar- ial loss (Goodfellow et al., 2014) is used in the latent space to discourage encoding style informa- tion in the latent representation. Although the dis- entangled latent representation brings better inter- pretability, in this paper, we address the following concerns for these models. 1) It is difficult to judge the quality of disen- tanglement. As reported in (Elazar and Goldberg, 2018; Lample et al., 2019), the style information can be still recovered from the latent represen- tation even the model has trained adversarially. Therefore, it is not easy to disentangle the stylistic property from the semantics of a sentence. …

Content truncated to 500 words