Unsupervised Text Style Transfer with Content Embeddings

Proceedings of Recent Advances in Natural Language Processing, pages 226–233 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_027 226 Unsupervised Text Style Transfer with Content Embeddings Keith Carlson Department of Computer Science Dartmouth College Hanover, NH 03755 keith.e.carlson.gr@dartmouth.edu Allen Riddell Department of Information and Library Science Indiana University Bloomington Bloomington, IN 47405 Daniel Rockmore Departments of Computer Science and Mathematics Dartmouth College Hanover, NH 03755 & The Santa Fe Institute Santa Fe, NM 87501 Abstract The style transfer task (here style is used in a broad “authorial” sense with many aspects in- cluding register, sentence structure, and vocab- ulary choice) takes text input and rewrites it in a specified target style preserving the mean- ing, but altering the style of the source text to match that of the target. Much of the existing research on this task depends on the use of par- allel datasets. In this work we employ recent results in unsupervised cross-lingual language modeling (XLM) and machine translation to effect style transfer while treating the input data as unaligned. First, we show that adding “content embeddings” to the XLM which cap- ture human-specified groupings of subject mat- ter can improve performance over the baseline model. Evaluation of style transfer has often relied on metrics designed for machine trans- lation which have received criticism of their suitability for this task. As a second contri- bution, we propose the use of a suite of clas- sical stylometrics as a useful complement for evaluation. We select a few such measures and include these in the analysis of our results. 1 Introduction In this paper we consider the problem of unsu- pervised holistic textual style transfer – both the accomplishment of the task, as well as its evalu- ation. The “style” of text is roughly the way in which a text communicates its content. It might be thought of as the “voice” characteristic of a given author, an emergent quality that encompasses a wide range of (more or less measurable) charac- teristics such as register, sentence structure, and vocabulary choice. Holistic style transfer takes a given text – written a priori in one “style” – and then rewrites it (preserving its content) in another style. Holistic style transfer is distinct from more narrow style modification techniques which ma- nipulate specific characteristics of prose such as formality, simplicity, or sentiment. To illustrate our idea of holistic style consider the following pair of translations of the opening lines of The Aeneid of Virgil. Arms and the man I sing, the first who came, Compelled by fate, an exile out of Troy...(Humphries et al., 1987) I sing of arms and the man who of old from the coasts of Troy came, an ex- ile of fate, to Italy and the shore of Lavinium...(Mackail, 1885) As another example compare verses from two different “versions” of a fixed verse from the Book of Genesis, the first from the King James Version And a river went out of Eden to water the garden; and from thence it was parted, and became …

Content truncated to 500 words