Semi-supervised Text Style Transfer: Cross Projection in Latent Space

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 4937–4946, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 4937 Semi-supervised Text Style Transfer: Cross Projection in Latent Space Mingyue Shang1∗, Piji Li2, Zhenxin Fu1, Lidong Bing4 Dongyan Zhao1,3, Shuming Shi2, Rui Yan1,3† 1Wangxuan Institute of Computer Technology, Peking University, China 2Tencent AI Lab, Shenzhen, China, 3Center for Data Science, China 4R&D Center Singapore, Machine Intelligence Technology, Alibaba DAMO Academy {shangmy, fuzhenxin, zhaodongyan, ruiyan}@pku.edu.cn {pijili, shumingshi}@tencent.com, l.bing@alibaba-inc.com Abstract Text style transfer task requires the model to transfer a sentence of one style to another style while retaining its original content meaning, which is a challenging problem that has long suffered from the shortage of parallel data. In this paper, we first propose a semi-supervised text style transfer model that combines the small-scale parallel data with the large-scale nonparallel data. With these two types of train- ing data, we introduce a projection function between the latent space of different styles and design two constraints to train it. We also in- troduce two other simple but effective semi- supervised methods to compare with. To eval- uate the performance of the proposed meth- ods, we build and release a novel style transfer dataset that alters sentences between the style of ancient Chinese poem and the modern Chi- nese. 1 Introduction Recently, the natural language generation (NLG) tasks have been attracting the growing atten- tion of researchers, including response genera- tion (Vinyals and Le, 2015), machine translation (Bahdanau et al., 2014), automatic summarization (Chopra et al., 2016), question generation (Gao et al., 2019), etc. Among these generation tasks, one interesting but challenging problem is text style transfer (Shen et al., 2017; Fu et al., 2018; Logeswaran et al., 2018). Given a sentence from one style domain, a style transfer system is re- quired to convert it to another style domain as well as keeping its content meaning unchanged. As a fundamental attribute of text, style can have a broad and ambiguous scope, such as ancient po- etry style v.s. modern language style and positive sentiment v.s. negative sentiment. ∗This work was done when Mingyue Shang was an intern at Tencent AI Lab. †Corresponding author: Rui Yan (ruiyan@pku.edu.cn) Building such a style transfer model has long suffered from the shortage of parallel training data, since constructing the parallel corpus that could align the content meaning of different styles is costly and laborious, which makes it difficult to train in a supervised way. Even some parallel cor- pora are built, they are still in a deficient scale for neural network based models. To tackle this issue, previous works utilize nonparallel data to train the model in an unsupervised way. One commonly used method is disentangling the style and con- tent from the source sentence (John et al., 2018; Shen et al., 2017; Hu et al., 2017). For the in- put, they learn the representations of styles and style-independent content, expecting …

Content truncated to 500 words