Collaborative Learning of Bidirectional Decoders for Unsupervised Text Style Transfer

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9250–9266 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 9250 Collaborative Learning of Bidirectional Decoders for Unsupervised Text Style Transfer Yun Ma1, Yangbin Chen2, Xudong Mao3, Qing Li1 1Department of Computing, The Hong Kong Polytechnic University 2Department of Information Engineering, The Chinese University of Hong Kong 3Department of Artificial Intelligence, Xiamen University mayun371@gmail.com, dongyiwu92@gmail.com xudong.xdmao@gmail.com, csqli@comp.polyu.edu.hk Abstract Unsupervised text style transfer aims to alter the underlying style of the text to a desired value while keeping its style-independent se- mantics, without the support of parallel training corpora. Existing methods struggle to achieve both high style conversion rate and low con- tent loss, exhibiting the over-transfer and under- transfer problems. We attribute these problems to the conflicting driving forces of the style con- version goal and content preservation goal. In this paper, we propose a collaborative learning framework for unsupervised text style trans- fer using a pair of bidirectional decoders, one decoding from left to right while the other de- coding from right to left. In our collaborative learning mechanism, each decoder is regular- ized by knowledge from its peer which has a different knowledge acquisition process. The difference is guaranteed by their opposite de- coding directions and a distinguishability con- straint. As a result, mutual knowledge dis- tillation drives both decoders to a better opti- mum and alleviates the over-transfer and under- transfer problems. Experimental results on two benchmark datasets show that our framework achieves strong empirical results on both style compatibility and content preservation. 1 Introduction Text style transfer is to transform an input text of a source style to a target style (i.e., style conversion goal) without loss of its style-independent infor- mation (i.e., content preservation goal). Concen- trating on different stylistic attributes, text style transfer has attracted much attention from vari- ous natural language processing applications, such as personalized machine translation (Rabinovich et al., 2017), text formalization (Zhang et al., 2020), and sentiment translation (Xu et al., 2018). Unfor- tunately, the parallel corpora with aligned input and output are usually unavailable, challenging the models to learn in an unsupervised manner. Input The dish is fresh and yummy. Expected Output The dish is old and disgusting. Over-Transfer The staff are rude! Under-Transfer The dish is old and yummy. Table 1: The over-transferred and the under-transferred results for an exemplar input in the postive→negative sentiment transfer task. One research line to address the unsupervised text style transfer task is to first disentangle the style-independent semantics (content) from the style-dependent semantics (style), and then pro- duce the output based on the disentangled content and the target style. The disentanglement is en- forced either implicitly (Hu et al., 2017; Shen et al., 2017; Fu et al., 2018; John et al., 2019), or explicitly (Li et al., 2018; Wu et al., 2019b; Xu et al., 2018; Madaan et al., 2020). Nevertheless, such disentan- glement has been discovered to be hardly met in practice (Elazar and Goldberg, 2018). Putting aside …

Content truncated to 500 words