A Hierarchical Reinforced Sequence Operation Method for Unsupervised Text Style Transfer

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4873–4883 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 4873 A Hierarchical Reinforced Sequence Operation Method for Unsupervised Text Style Transfer Chen Wu1∗, Xuancheng Ren2∗, Fuli Luo2, Xu Sun2,3 1Department of Foreign Languages and Literatures, Tsinghua University 2MOE Key Laboratory of Computational Linguistics, School of EECS, Peking University 3Center for Data Science, Beijing Institute of Big Data Research, Peking University wu-c16@mails.tsinghua.edu.cn {renxc, luofuli, xusun}@pku.edu.cn Abstract Unsupervised text style transfer aims to al- ter text styles while preserving the content, without aligned data for supervision. Exist- ing seq2seq methods face three challenges: 1) the transfer is weakly interpretable, 2) gener- ated outputs struggle in content preservation, and 3) the trade-off between content and style is intractable. To address these challenges, we propose a hierarchical reinforced sequence op- eration method, named Point-Then-Operate (PTO), which consists of a high-level agent that proposes operation positions and a low- level agent that alters the sentence. We pro- vide comprehensive training objectives to con- trol the fluency, style, and content of the out- puts and a mask-based inference algorithm that allows for multi-step revision based on the single-step trained agents. Experimental re- sults on two text style transfer datasets show that our method significantly outperforms re- cent methods and effectively addresses the aforementioned challenges. 1 1 Introduction Text style transfer aims to convert a sentence of one style into another while preserving the style-independent content (Shen et al., 2017; Fu et al., 2018). In most cases, aligned sentences are not available, which requires learning from non- aligned data. Previous work mainly learns dis- entangled content and style representations using seq2seq (Sutskever et al., 2014) models and de- composes the transfer into neutralization and styl- ization steps. Although impressive results have been achieved, three challenges remain: 1) the in- terpretability of the transfer procedure is still weak in seq2seq models, 2) generated sentences are usu- ally highly stylized with poor content preserva- ∗Equal Contributions. 1 Our code is available at https://github.com/ ChenWu98/Point-Then-Operate. I will be going back and enjoying this great place ! I will be going back and enjoying this horrible place ! I will be going back and avoid this horrible place ! I will not be going back and avoid this horrible place ! Replace(great,horrible) Replace(enjoying,avoid) InsertBefore(be,not) [Input] [Iteration 1] [Iteration 2] [Iteration 3] Figure 1: Our proposed Point-Then-Operate (PTO) ap- plied to a real test sample. A high-level agent (red squares) iteratively proposes operation positions, and a low-level agent (arrows) alters the sentence based on the high-level proposals. Compared with seq2seq methods, PTO is more interpretable and better pre- serves style-independent contents. tion, and 3) the trade-off between content preser- vation and style polarity is intractable. To address these challenges, we propose a se- quence operation-based method within the hierar- chical reinforcement learning (HRL) framework, named Point-Then-Operate (PTO). It consists of a hierarchy of a high-level agent that proposes operation positions and a low-level agent …

Content truncated to 500 words