Multi-Style Transfer with Discriminative Feedback on Disjoint Corpus

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3500–3510 June 6–11, 2021. ©2021 Association for Computational Linguistics 3500 Multi-Style Transfer with Discriminative Feedback on Disjoint Corpus Navita Goyal, Balaji Vasan Srinivasan, Anandhavelu N, Abhilasha Sancheti Adobe Research, India {navgoyal, balsrini, anandvn, sancheti}@adobe.com Abstract Style transfer has been widely explored in nat- ural language generation with non-parallel cor- pus by directly or indirectly extracting a no- tion of style from source and target domain corpus. A common shortcoming of existing approaches is the prerequisite of joint anno- tations across all the stylistic dimensions un- der consideration. Availability of such dataset across a combination of styles limits the exten- sion of these setups to multiple style dimen- sions. While cascading single-dimensional models across multiple styles is a possibility, it suffers from content loss, especially when the style dimensions are not completely inde- pendent of each other. In our work, we re- lax this requirement of jointly annotated data across multiple styles by using independently acquired data across different style dimensions without any additional annotations. We initial- ize an encoder-decoder setup with transformer- based language model pre-trained on a generic corpus and enhance its re-writing capability to multiple target style dimensions by employing multiple style-aware language models as dis- criminators. Through quantitative and quali- tative evaluation, we show the ability of our model to control styles across multiple style di- mensions while preserving content of the input text. We compare it against baselines involv- ing cascaded state-of-the-art uni-dimensional style transfer models. 1 Introduction Style transfer is a popular task in natural language processing and has been studied on attributes like age or gender (Subramanian et al., 2018), styles emanating from social construct like formality (Rao and Tetreault, 2018) and politeness (Madaan et al., 2020), linguistic styles based on author writ- ing style (Syed et al., 2020), or psycho-linguistic styles based on personality types (Mairesse and Walker, 2011). While early style transfer frame- works were modeled as a supervised learning task on a parallel corpus, state-of-the-art models are semi-supervised/unsupervised and operate on non- parallel corpus. These models achieve style trans- fer by aligning source and target distribution of sentences from non-parallel corpus (Shen et al., 2017), disentangling content space from style space in latent representation (Hu et al., 2017) or em- ploying self-reconstruction (Dai et al., 2019) and back translation (Lample et al., 2018) objectives to achieve pseudo-supervision with non-parallel corpus. Recent works have also modeled this in a self-supervised manner where rewriting (trans- fer) is achieved by utilizing corpus from the target style alone (Syed et al., 2020). These wide stud- ies have also led to the curation and benchmark- ing of non-parallel dataset for various style dimen- sions, such as sentiment (Li et al., 2018), formality (Rao and Tetreault, 2018), politeness (Danescu- Niculescu-Mizil et al., 2013), excitement (Sancheti et al., 2020), etc. But availability of data with joint tagging across multiple styles is limited and has restricted the ability of …

Content truncated to 500 words