Controllable Natural Language Generation with Contrastive Prefixes

Findings of the Association for Computational Linguistics: ACL 2022, pages 2912 - 2924 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Controllable Natural Language Generation with Contrastive Prefixes Jing Qian1, Li Dong2, Yelong Shen2, Furu Wei2, Weizhu Chen2 1University of California, Santa Barbara 2Microsoft Corporation jing_qian@cs.ucsb.edu {lidong1,yeshe,fuwei,wzchen}@microsoft.com Abstract To guide the generation of large pretrained language models (LM), previous work has focused on directly fine-tuning the language model or utilizing an attribute discriminator. In this work, we propose a novel lightweight framework for controllable GPT2 (Radford et al., 2019) generation, which utilizes a set of small attribute-specific vectors, called pre- fixes (Li and Liang, 2021), to steer natural lan- guage generation. Different from Li and Liang (2021), where each prefix is trained indepen- dently, we take the relationship among prefixes into consideration and train multiple prefixes simultaneously, as illustrated in Figure 1. We propose a novel supervised method and also an unsupervised method to train the prefixes for single-aspect control while the combination of these two methods can achieve multi-aspect control. Experimental results on both single- aspect and multi-aspect control show that our methods can guide generation towards the de- sired attributes while keeping high linguistic quality. 1 Introduction The goal of controllable Natural Language Genera- tion (NLG) is to guide generation towards the de- sired attributes in the concerned aspects of the text. For example, the aspect can be topic or sentiment, and sentiment may have two attributes: positive and negative. Previous work has focused on directly fine-tuning the existing models (Keskar et al., 2019; Hu et al., 2017; Ficler and Goldberg, 2017) or using a discriminator to guide generation (Dathathri et al., 2020; Krause et al., 2020; Holtzman et al., 2018). CTRL (Keskar et al., 2019) achieves controllabil- ity at the expense of training a large conditional LM. GeDi (Krause et al., 2020) also trains con- ditional LMs but uses them as discriminators to guide generation, introducing additional 345M pa- rameters. Besides, GeDi focuses on single-aspect control, ignoring the need for multi-aspect control. Figure 1: A comparison of prefix-tuning (Li and Liang, 2021) (top) and our framework (bottom) on sentiment control. The solid arrows show the training process, while the dashed ones show the inference (generation) process. In our proposed framework, the training can be supervised, semi-supervised, or unsupervised. PPLM (Dathathri et al., 2020) guides generation by iteratively updating the LM’s hidden activations. However, this decoding strategy is extremely com- putationally intensive, resulting in a slow genera- tion speed (Gehman et al., 2020). Prefix-tuning (Li and Liang, 2021) proposes to optimize a prefix, which is a small continuous task- specific vector, as a lightweight alternative to fine- tuning an NLG task, such as table-to-text genera- tion or summarization. Inspired by Li and Liang (2021), we propose to use prefixes, a set of small continuous attribute-specific vectors, to steer NLG. Compared with using an attribute model or a gener- ative discriminator (Dathathri et al., 2020; Krause et al., 2020), using learned prefixes to achieve con- trollability has the following …

Content truncated to 500 words