DisCup: Discriminator Cooperative Unlikelihood Prompt-tuning for Controllable Text Generation

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3392 - 3406 December 7-11, 2022 ©2022 Association for Computational Linguistics DisCup: Discriminator Cooperative Unlikelihood Prompt-tuning for Controllable Text Generation Hanqing Zhang Beijing Institute of Technology, Beijing, China zhanghanqing@bit.edu.cn Dawei Song∗ Beijing Institute of Technology, Beijing, China dwsong@bit.edu.cn Abstract Prompt learning with immensely large Casual Language Models (CLMs) has been shown promising for attribute-controllable text gen- eration (CTG). However, vanilla prompt tun- ing tends to imitate training corpus characteris- tics beyond the control attributes, resulting in a poor generalization ability. Moreover, it is less able to capture the relationship between different attributes, further limiting the con- trol performance. In this paper, we propose a new CTG approach, namely DisCup, which incorporates the attribute knowledge of discrim- inator to optimize the control-prompts, steer- ing a frozen CLM to produce attribute-specific texts. Specifically, the frozen CLM model, capable of producing multitudinous texts, is first used to generate the next-token candidates based on the context, so as to ensure the di- versity of tokens to be predicted. Then, we leverage an attribute-discriminator to select de- sired/undesired tokens from those candidates, providing the inter-attribute knowledge. Fi- nally, we bridge the above two traits by an unlikelihood objective for prompt-tuning. Ex- tensive experimental results show that DisCup can achieve a new state-of-the-art control per- formance while maintaining an efficient and high-quality text generation, only relying on around 10 virtual tokens1. 1 Introduction Attribute-controllable text generation (CTG) aims to produce texts that satisfy desired attributes (e.g., sentiment, topic, etc.), facilitating safer and more practical text generation applications. For exam- ple, we need to control the emotion and polite- ness of generated text in a dialogue system for a more friendly interaction, while it is also crucial to avoid generating mindless and offensive con- tent such as racial discrimination and toxic words. ∗Corresponding author, also with The Open University, United Kingdom. 1The code implementation is available at: https://github.com/littlehacker26/disc-cooperative-up-tuning Current Transformer-based pre-trained language models (PLMs), especially casual language models (CLMs) like the GPT family (Brown et al., 2020), have enabled generation of texts of an unprece- dented quality. However, due to the lack of in- terpretability of deep neural networks, it is often hard to guarantee the controllability of these mod- els (Zhang et al., 2022). This is a challenging and largely unsolved problem. The most natural ways of using transformer- based PLMs for CTG are to fine-tune or retrain the models (Chan et al., 2021; Keskar et al., 2019; Khalifa et al., 2021; Gururangan et al., 2020a), so as to control the PLMs to generate texts sat- isfying the control attributes. Such methods have achieved certain performance breakthroughs in this area. Nevertheless, the scale of PLMs is getting larger in recent years, making the PLMs resource- intensive to fine-tune or retrain. Therefore, increas- ing attention has been paid to the decoding-time methods, where a PLM is always fixed and a guided module is used to steer the text generation process. The core idea of …

Content truncated to 500 words