DRAG: Director-Generator Language Modelling Framework for Non-Parallel Author Stylized Rewriting

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 863–873 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 863 DRAG: Director-Generator Language Modelling Framework for Non-Parallel Author Stylized Rewriting Hrituraj Singh Adobe Research hrisingh@adobe.com Gaurav Verma∗ Georgia Tech gverma@gatech.edu Aparna Garimella Adobe Research garimell@adobe.com Balaji Vasan Srinivasan Adobe Research balsrini@adobe.com Abstract Author stylized rewriting is the task of rewrit- ing an input text in a particular author’s style. Recent works in this area have leveraged Transformer-based language models in a de- noising autoencoder setup to generate author stylized text without relying on a parallel cor- pus of data. However, these approaches are limited by the lack of explicit control of target attributes and being entirely data-driven. In this paper, we propose a Director-Generator framework to rewrite content in the target au- thor’s style, specifically focusing on certain target attributes. We show that our proposed framework works well even with a limited- sized target author corpus. Our experiments on corpora consisting of relatively small-sized text authored by three distinct authors show significant improvements upon existing works to rewrite input texts in target author’s style. Our quantitative and qualitative analyses fur- ther show that our model has better meaning re- tention and results in more fluent generations. 1 Introduction With recent advances in language modeling tech- niques that have resulted in powerful language mod- els (Radford et al., 2019; Devlin et al., 2018; Brown et al., 2020) along with an increased interest in styl- ized text generation (Hu et al., 2017; Shen et al., 2017; Subramanian et al., 2018; Fu et al., 2018; Niu and Bansal, 2018), large language models have been successfully tuned to achieve text stylization (Lample et al., 2018; Ziegler et al., 2019; Syed et al., 2020; Singh et al., 2020). Apart from trans- ferring an input text to the target style, which has received recent interest from the community, un- derstanding and measuring style have been persis- tently explored over the last few decades (Kessler ∗This was work was carried out when the author was at Adobe Research. et al., 1997; Garera and Yarowsky, 2009; Liu, 2012; Verma and Srinivasan, 2019). Lying at the intersec- tion of style transfer enabled by advanced language models and a deep understanding of style as a nu- anced combination of several linguistic concepts, problems like stylized generation or stylized rewrit- ing have gained further traction. A large body of work in style transfer focuses on binary aspects such as positive-negative sentiment (Li et al., 2018; Ziegler et al., 2019), formal-informal (Jain et al., 2019), and sometimes a mixture of these attributes (Subramanian et al., 2018). To fuel this interest in such binary stylization, some datasets compris- ing of text from the extreme ends of these spec- trums have also emerged (e.g., positive-negative sentiments (Mathews et al., 2016), formal-informal (Rao and Tetreault, 2018)). As pointed by Syed et al. (2020), author stylized rewriting does not di- rectly fit under any of these variants as …

Content truncated to 500 words