Generic resources are what you need: Style transfer tasks without task-specific parallel training data

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4241–4254 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 4241 Generic resources are what you need: Style transfer tasks without task-specific parallel training data Huiyuan Lai, Antonio Toral, Malvina Nissim CLCG, University of Groningen / The Netherlands {h.lai, a.toral.ruiz, m.nissim}@rug.nl Abstract Style transfer aims to rewrite a source text in a different target style while preserving its content. We propose a novel approach to this task that leverages generic resources, and without using any task-specific parallel (source–target) data outperforms existing un- supervised approaches on the two most popu- lar style transfer tasks: formality transfer and polarity swap. In practice, we adopt a multi- step procedure which builds on a generic pre- trained sequence-to-sequence model (BART). First, we strengthen the model’s ability to rewrite by further pre-training BART on both an existing collection of generic paraphrases, as well as on synthetic pairs created using a general-purpose lexical resource. Second, through an iterative back-translation approach, we train two models, each in a transfer direc- tion, so that they can provide each other with synthetically generated pairs, dynamically in the training process. Lastly, we let our best re- sulting model generate static synthetic pairs to be used in a supervised training regime. Be- sides methodology and state-of-the-art results, a core contribution of this work is a reflection on the nature of the two tasks we address, and how their differences are highlighted by their response to our approach. 1 Introduction Text style transfer is, broadly put, the task con- verting a text of one style into another while pre- serving its content. In its recent tradition within Natural Language Generation (NLG), two tasks and their corresponding datasets have been com- monly used (Zhang et al., 2018; Luo et al., 2019; Wu et al., 2019; Yi et al., 2020; Zhou et al., 2020). One dataset was specifically created for formality transfer and contains parallel data (GYAFC (Rao and Tetreault, 2018)), while the other one contains a large amount of non-parallel sentiment labelled texts (YELP (Li et al., 2018)), with parallel pairs for test, and is used for the task of polarity swap. Examples from these datasets are shown in Table 1. The two tasks are usually conflated in the lit- erature under the general style transfer label and addressed with the same methods, but we find this an oversimplification. Formality transfer implies rewriting a formal sentence into its informal coun- terpart (or viceversa) while preserving its meaning. Polarity swap, instead, aims to change a positive text into a negative one (or viceversa); and while the general theme must be preserved, the meaning is by definition not maintained (e.g. “I hated that film” → “I loved that film”). In line with previous work, we also address both tasks in a similar way, but this is actually to unveil how their different nature affects modelling and evaluation. Due to the general scarcity of parallel data, previous works mainly adopted unsupervised ap- proaches, dubbed …

Content truncated to 500 words