Deep Learning for Text Style Transfer: A Survey

Deep Learning for Text Style Transfer: A Survey Di Jin∗ Amazon Alexa AI djinamzn@amazon.com Zhijing Jin* Max Planck Institute for Intelligent Systems Empirical Inference Department and ETH Z¨urich Department of Computer Science zjin@tue.mpg.de Zhiting Hu UC San Diego Halıcıo˘glu Data Science Institute (HDSI) zhh019@ucsd.edu Olga Vechtomova University of Waterloo Faculty of Engineering ovechtom@uwaterloo.ca Rada Mihalcea University of Michigan EECS, College of Engineering mihalcea@umich.edu Text style transfer is an important task in natural language generation, which aims to control certain attributes in the generated text, such as politeness, emotion, humor, and many others. It has a long history in the field of natural language processing, and recently has re-gained significant attention thanks to the promising performance brought by deep neural models. In this article, we present a systematic survey of the research on neural text style transfer, spanning over 100 representative articles since the first neural text style transfer work in 2017. We discuss the ∗ Equal contribution. Submission received: 25 April 2021; revised version received: 30 August 2021; accepted for publication: 4 December 2021. https://doi.org/10.1162/COLI a 00426 © 2022 Association for Computational Linguistics Published under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license Computational Linguistics Volume 48, Number 1 task formulation, existing datasets and subtasks, evaluation, as well as the rich methodologies in the presence of parallel and non-parallel data. We also provide discussions on a variety of important topics regarding the future development of this task.1 1. Introduction Language is situational. Every utterance fits in a specific time, place, and scenario, conveys specific characteristics of the speaker, and typically has a well-defined intent. For example, someone who is uncertain is more likely to use tag questions (e.g., “This is true, isn’t it?”) than declarative sentences (e.g., “This is definitely true.”). Similarly, a professional setting is more likely to include formal statements (e.g., “Please consider taking a seat.”) as compared to an informal situation (e.g., “Come and sit!”). For artificial intelligence systems to accurately understand and generate language, it is necessary to model language with style/attribute,2 which goes beyond merely verbalizing the semantics in a non-stylized way. The values of the attributes can be drawn from a wide range of choices depending on pragmatics, such as the extent of formality, politeness, simplicity, personality, emotion, partner effect (e.g., reader awareness), genre of writing (e.g., fiction or non-fiction), and so on. The goal of TST is to automatically control the style attributes of text while preserv- ing the content. TST has a wide range of applications, as outlined by McDonald and Pustejovsky (1985) and Hovy (1987). The style of language is crucial because it makes natural language processing more user-centered. TST has many immediate applications. For instance, one such application is intelligent bots, for which users prefer distinct and consistent persona (e.g., empathetic) instead of emotionless or inconsistent persona. Another application is the development of intelligent writing assistants; for example, non-expert writers often need to polish their writing to better fit their purpose, for example, more professional, polite, objective, humorous, …

Content truncated to 500 words