Text Style Transfer with Contrastive Transfer Pattern Mining

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7914–7927 July 9-14, 2023 ©2023 Association for Computational Linguistics Text Style Transfer with Contrastive Transfer Pattern Mining Jingxuan Han1, Quan Wang2, Licheng Zhang1, Weidong Chen1 Yan Song1 and Zhendong Mao1∗ 1University of Science and Technology of China, Hefei, China 2MOE Key Laboratory of Trustworthy Distributed Computing and Service, Beijing University of Posts and Telecommunications, Beijing, China hjx999222@mail.ustc.edu.cn, zdmao@ustc.edu.cn Abstract Text style transfer (TST) is an important task in natural language generation, which aims to alter the stylistic attributes (e.g., sentiment) of a sentence and keep its semantic meaning un- changed. Most existing studies mainly focus on the transformation between styles, yet ig- nore that this transformation can be actually carried out via different hidden transfer pat- terns. To address this problem, we propose a novel approach, contrastive transfer pattern mining (CTPM), which automatically mines and utilizes inherent latent transfer patterns to improve the performance of TST. Specifically, we design an adaptive clustering module to au- tomatically discover hidden transfer patterns from the data, and introduce contrastive learn- ing based on the discovered patterns to obtain more accurate sentence representations, and thereby benefit the TST task. To the best of our knowledge, this is the first work that pro- poses the concept of transfer patterns in TST, and our approach can be applied in a plug-and- play manner to enhance other TST methods to further improve their performance. Extensive experiments on benchmark datasets verify the effectiveness and generality of our approach.1 1 Introduction Text style transfer (TST), an important task in natu- ral language generation, aims to change the stylistic properties while preserving the style-independent content within the context. Stylistic properties in- clude but are not limited to sentiment, politeness, formality, and humor. This task has wide applica- tions, such as neutralizing offensive remarks (Ci- cero et al., 2018), data augmentation (Xu et al., 2019), and human-computer interaction (Li et al., 2016). The main difficulty in this task is the lack of parallel datasets, making most previous works develop their methods in an unsupervised man- ner and can be roughly divided into two groups. ∗Corresponding author: Zhendong Mao. 1Our code and data will be released at GitHub. Figure 1: Three cases from the test set of sentiment transfer dataset Yelp, which can reflect different transfer patterns. Methods in the first group (Lee et al., 2021; Reid and Zhong, 2021) separate style-independent sen- tence representations and revise them with style attributes, while methods in the second group (Dai et al., 2019; Kashyap et al., 2022) directly revise an entangled representation of an input by using an extra style embedding. However, the two groups of existing methods only focus on the transformation between styles and do not take into account that this transformation might be achieved via different latent transfer patterns. As a matter of fact, there are lots of transfer patterns in the TST task. Figure 1 shows three cases from the test set of …

Content truncated to 500 words