StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2116–2138 June 6–11, 2021. ©2021 Association for Computational Linguistics 2116 STYLEPTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer Yiwei Lyu♡∗, Paul Pu Liang♡∗, Hai Pham♠∗, Eduard Hovy♠, Barnabás Póczos♡, Ruslan Salakhutdinov♡, Louis-Philippe Morency♠ ♡Machine Learning Department, Carnegie Mellon University ♠Language Technologies Institute, Carnegie Mellon University {ylyu1,pliang,htpham}@cs.cmu.edu https://github.com/lvyiwei1/StylePTB/ Abstract Text style transfer aims to controllably gener- ate text with targeted stylistic changes while maintaining core meaning from the source sen- tence constant. Many of the existing style transfer benchmarks primarily focus on indi- vidual high-level semantic changes (e.g. posi- tive to negative), which enable controllability at a high level but do not offer fine-grained control involving sentence structure, empha- sis, and content of the sentence. In this pa- per, we introduce a large-scale benchmark, STYLEPTB, with (1) paired sentences under- going 21 fine-grained stylistic changes span- ning atomic lexical, syntactic, semantic, and thematic transfers of text, as well as (2) com- positions of multiple transfers which allow modeling of fine-grained stylistic changes as building blocks for more complex, high-level transfers. By benchmarking existing meth- ods on STYLEPTB, we find that they strug- gle to model fine-grained changes and have an even more difficult time composing mul- tiple styles. As a result, STYLEPTB brings novel challenges that we hope will encourage future research in controllable text style trans- fer, compositional models, and learning dis- entangled representations. Solving these chal- lenges would present important steps towards controllable text generation. 1 Introduction At the heart of interactive AI systems lies the el- ement of communication as a channel to convey intentions using different stylistic attributes. Re- search in human-AI interaction has focused on building dialog systems (Celikyilmaz et al., 2018), virtual assistants (Cooper et al., 2004), and intelli- gent agents (Kim et al., 2013; Liang et al., 2020a; Pittermann et al., 2010) that can communicate their intentions with specific styles for different situa- tions, target audiences, and environments (Lample ∗authors contributed equally The bad service of the waitresses make me dread going sometimes. The good service of the waitresses makes me dread going sometimes. The good service of the waitresses makes me enjoy going sometimes. I left three messages without a call back. I left three messages. I left three thankful messages. After 3 months they can't be too new now. After 3 months they can't be too new. (adj antonym replacement) (verb antonym replacement) (info addition) (PP removal) (adv removal) Negative Positive Informal Formal Morgan Freeman did the new one. Morgan Freeman did perform the new one. The new one was performed by Morgan Freeman. (info addition) (active to passive) They can't be too new after 3 months. (PP front to back) High-level style transfer Source styles Target styles Fine-grained compositional style transfer Figure 1: STYLEPTB provides a large-scale resource to study fine-grained compositional style transfer. The styles provided in STYLEPTB (in green) span lexical, syntax, semantic, and thematic aspects (DiMarco …

Content truncated to 500 words