Text Style Transfer via Optimal Transport

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2532 - 2541 July 10-15, 2022 ©2022 Association for Computational Linguistics Text Style Transfer via Optimal Transport Nasim Nouri Raouf Medical Group Tehran, Iran nasimnouri@raoufmed.com Abstract Text style transfer (TST) is a well-known task whose goal is to convert the style of the text (e.g., from formal to informal) while preserv- ing its content. Recently, it has been shown that both syntactic and semantic similarities between the source and the converted text are important for TST. However, the interaction between these two concepts has not been mod- eled. In this work, we propose a novel method based on Optimal Transport for TST to simulta- neously incorporate syntactic and semantic in- formation into similarity computation between the source and the converted text. We eval- uate the proposed method in both supervised and unsupervised settings. Our analysis reveal the superiority of the proposed model in both settings. 1 Introduction Text style transfer (TST) is an important task in NLP that aims to change the style of a given text from source style to target style (e.g., formal to informal) while preserving its content. For instance, the formal sentence “However, I do believe it to be punk" is converted to the informal equivalent sentence “I’d say it is punk though". This task could be helpful for downstream applications such as text simplification, information extraction, and question answering. Due to the importance of TST, this task has been approached with different techniques ranging from feature-based models (Xu et al., 2012) to recent ad- vanced deep learning solutions (Chen et al., 2018; Lee et al., 2021a; Huang et al., 2021). The recent work can be categorized as supervised (i.e., parallel corpus with sentences in source and target style) (Lai et al., 2021), unsupervised (i.e., sentences in source and target style are available but they are not aligned) (Krishna et al., 2020), or semi-supervised (combination of parallel and non-aligned corpora) (Chawla and Yang, 2020) methods. The three crit- ical objectives of any TST system are to (1) gen- erate a text in the target style, (2) keep the con- tent of the source text, and (3) generate fluent sen- tences (Krishna et al., 2020). It has been shown that fine-tuning transformer-based language models on each of these objectives (i.e., using Reinforcement Learning) can achieve promising results (Lai et al., 2021; Liu et al., 2021). However, one of the lim- itations of the existing works is that the content preservation (i.e., the second objective) is fulfilled at either the surface-form level (i.e., by encouraging the same words to appear in both texts) (Lai et al., 2021) or at the semantics level (i.e., by encourag- ing high mutual information between the two texts) (Chawla and Yang, 2020); ignoring the role of syn- tactic information. Syntactic information (e.g., de- pendency tree) can be used to explicitly encode the connections between the words of the sentence, thereby playing an important …

Content truncated to 500 words