LEWIS: Levenshtein Editing for Unsupervised Text Style Transfer

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3932–3944 August 1–6, 2021. ©2021 Association for Computational Linguistics 3932 LEWIS: Levenshtein Editing for Unsupervised Text Style Transfer Machel Reid∗ The University of Tokyo machelreid@weblab.t.u-tokyo.ac.jp Victor Zhong University of Washington vzhong@cs.washington.edu Abstract Many types of text style transfer can be achieved with only small, precise edits (e.g. sentiment transfer from I had a terrible time... to I had a great time...). We propose a coarse-to-fine editor for style transfer that transforms text using Levenshtein edit operations (e.g. insert, replace, delete). Unlike prior single-span edit methods, our method concurrently edits multiple spans in the source text. To train without parallel style text pairs (e.g. pairs of +/- sentiment statements), we propose an un- supervised data synthesis procedure. We first convert text to style-agnostic templates using style classifier attention (e.g. I had a SLOT time...), then fill in slots in these templates using fine-tuned pretrained language models. Our method outperforms existing genera- tion and editing style transfer methods on sentiment (YELP, AMAZON) and politeness (POLITE) transfer. In particular, multi-span editing achieves higher performance and more diverse output than single-span editing. Moreover, compared to previous methods on unsupervised data synthesis, our method results in higher quality parallel style pairs and improves model performance.1 1 Introduction In text style transfer, a model changes the style of a source text (e.g. sentiment, politeness) into a target style, while otherwise changing as little as possible about the input. Many types of style transfer can be performed with only small, precise edits instead of generation from scratch. Consider the task of transforming a negative sentiment sentence such as the worst ribs I’ve ever had! to a posi- tive sentence such as probably the best ribs ∗Corresponding author 1Code and data can be found at https://github. com/machelreid/lewis ever!. Here, we need only invert the negative sentiment phrase around worst — the references to ribs should be left as-is. Recent and concur- rent work on text style transfer propose single-span editing (for insertion and replacement) (Wu et al., 2019; Malmi et al., 2020) as an alternative to gen- erating the target text from scratch (Prabhumoye et al., 2018; He et al., 2020b; John et al., 2019; Shen et al., 2017; Fu et al., 2018). We introduce a more flexible and powerful multi- span editing method that identifies multiple style- specific components of the text and concurrently edits them into the target style. Given a source text, we first predict the sequence of coarse-grain Levenshtein edit types (e.g. insert, replace, delete) that transform the source text to the target text, then fill insertion and replacement edits using a generator. In the previous example, the operations correspond to inserting the word probably before the, replacing worst with best, and removing the words I’ve and had. This example is illustrated in detail in Figure 1. Learning to edit requires supervised source- target text pairs. How do we learn high-quality editors when no such supervised parallel data ex- ists? Given a style text, …

Content truncated to 500 words