Transductive Learning for Unsupervised Text Style Transfer

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2510–2521 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 2510 Transductive Learning for Unsupervised Text Style Transfer Fei Xiao1,4, Liang Pang1∗, Yanyan Lan3∗, Yan Wang5, Huawei Shen1,4, Xueqi Cheng2,4 1Data Intelligence System Research Center and 2CAS Key Lab of Network Data Science and Technology, Institute of Computing Technology, Chinese Academy of Sciences 3Institute for AI Industry Research, Tsinghua University 4University of Chinese Academy of Sciences 5Tencent AI Lab {xiaofei19s, pangliang, shenhuawei, cxq}@ict.ac.cn lanyanyan@tsinghua.edu.cn, brandenwang@tencent.com Abstract Unsupervised style transfer models are mainly based on an inductive learning approach, which represents the style as embeddings, decoder parameters, or discriminator parameters and directly applies these general rules to the test cases. However, the lacking of parallel corpus hinders the ability of these inductive learning methods on this task. As a result, it is likely to cause severe inconsistent style expressions, like the salad is rude. To tackle this problem, we propose a novel transductive learning ap- proach in this paper, based on a retrieval-based context-aware style representation. Specifi- cally, an attentional encoder-decoder with a retriever framework is utilized. It involves top- K relevant sentences in the target style in the transfer process. In this way, we can learn a context-aware style embedding to alleviate the above inconsistency problem. In this pa- per, both sparse (BM25) and dense retrieval functions (MIPS) are used, and two objective functions are designed to facilitate joint learn- ing. Experimental results show that our method outperforms several strong baselines. The pro- posed transductive learning approach is general and effective to the task of unsupervised style transfer, and we will apply it to the other two typical methods in the future. 1 Introduction Text style transfer is an essential topic of natural language generation, which is widely used in many tasks such as sentiment transfer (Hu et al., 2017; Shen et al., 2017), dialogue generation (Zhou et al., 2018; Niu and Bansal, 2018; Su et al., 2021), and text formalization (Jain et al., 2019). The target is to change the style of the text while retaining style-independent content. As it is usually hard to obtain large parallel corpora with the same content *Corresponding Author Fantastic Delicious Great Cold Rude Terrible The salad is delicious. The salad is rude. ? Inductive Learning Transductive Learning Unpaired Inconsistent Problem! The sandwiches are terrible. Specific Case The salad is terrible. Rules Figure 1: Illustration of the inconsistency problem and the idea of our transductive learning approach. and different styles, unsupervised text style transfer becomes a hot yet challenging research topic in recent years. Most existing methods in this area try to find the general style transfer rules with an inductive learn- ing paradigm, where style is represented as a spe- cific form, e.g., embeddings, decoder parameters, or classifier parameters. For example, embedding based methods (Shen et al., 2017; Lample et al., 2019; John et al., 2019; Dai et al., 2019; Yi et al., 2020; He et al., 2020) utilize a highly …

Content truncated to 500 words