Syntax Matters! Syntax-Controlled in Text Style Transfer

Proceedings of Recent Advances in Natural Language Processing, pages 566–575 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_064 566 Syntax Matters! Syntax-Controlled in Text Style Transfer Zhiqiang Hu University of Electronic Science and Technology of China hzq950419@gmail.com Roy Ka-Wei Lee Singapore University of Technology and Design roy lee@sutd.edu.sg Charu C. Aggarwal IBM T. J. Watson Research Center charu@us.ibm.com Abstract Existing text style transfer (TST) methods rely on style classifiers to disentangle the text’s con- tent and style attributes for text style trans- fer. While the style classifier plays a criti- cal role in existing TST methods, there is no known investigation on its effect on the TST methods. In this paper, we conduct an em- pirical study on the limitations of the style classifiers used in existing TST methods. We demonstrate that the existing style classifiers cannot learn sentence syntax effectively and ultimately worsen existing TST models’ per- formance. To address this issue, we propose a novel Syntax-Aware Controllable Genera- tion (SACG) model, which includes a syntax- aware style classifier that ensures learned style latent representations effectively capture the syntax information for TST. Through exten- sive experiments on two popular TST tasks, we show that our proposed method signifi- cantly outperforms the state-of-the-art meth- ods. Our case studies have also demonstrated SACG’s ability to generate fluent target-style sentences that preserved the original content. 1 Introduction Text Style Transfer (TST) is an increasingly pop- ular natural language generation task that aims to change the stylistic properties (e.g., the senti- ment of text) of the text while retaining its style- independent content (Hu et al., 2020). Due to the difficulty in obtaining training sentence pairs with the same content and differing styles, most existing methods are designed to perform TST in an unsu- pervised manner; the models only have access to non-parallel, but style-labelled sentences. A popular TST approach is to leverage an ad- versarial learning autoencoder framework where a style classifier or discriminator is pre-trained to first disentangle the content and style latent repre- sentations, before using a decoder to generate the output sentence in the target style (Shen et al., 2017; Zhao et al., 2018; Fu et al., 2018; Chen et al., 2018). Another line of work proposed attribute-controlled generation methods where the style attribute latent vector is learned and combine with the latent repre- sentation of the text to generate output sentences in target style (Hu et al., 2017; Dai et al., 2019; Zhang et al., 2018a). Similar to the adversarial learning approach, the learning of the style attribute latent vector is guided using a pre-trained style classifier. A common key component in the two aforemen- tioned TST approaches is the usage of a style clas- sifier. However, little is known about the effects of the style classifier on these models. For instance, is the style classifier effective in learning the style in the text? What aspects of the text style has the exist- ing style classifier learned? Can the style classifiers distinguish text’s syntax? Can the style classifier guide TST models to …

Content truncated to 500 words