Multi-Attribute Controlled Text Generation with Contrastive-Generator and External-Discriminator

Proceedings of the 29th International Conference on Computational Linguistics, pages 5904–5913 October 12–17, 2022. 5904 Multi-Attribute Controlled Text Generation with Contrastive-Generator and External-Discriminator Guisheng Liu1, Yi Li1∗, Yanqing Guo1 , Xiangyang Luo2, Bo Wang1 1Dalian University of Technology 2Information Engineering University lgs0000@mail.dlut.edu.cn, {liyi, guoyq}@dlut.edu.cn xiangyangluo@126.com, bowang@dlut.edu.cn Abstract Though existing researches have achieved im- pressive results in controlled text generation, they focus mainly on single-attribute control. However, in applications like automatic com- ments, the topic and sentiment need to be con- trolled simultaneously. In this work, we pro- pose a new framework for multi-attribute con- trolled text generation. To achieve this, we design a contrastive-generator that can effec- tively generate texts with more attributes. In order to increase the convergence of the text on the desired attributes, we adopt an external- discriminator to distinguish whether the gen- erated text holds the desired attributes. More- over, we propose top-n weighted decoding to further improve the relevance of texts to at- tributes. Automated evaluations and human evaluations show that our framework achieves remarkable controllability in multi-attribute generation while keeping the text fluent and diverse. It also yields promising performance on zero-shot generation. 1 Introduction Large pre-trained language models (LM) have achieved impressive performance in natural lan- guage generation tasks (Radford et al., 2018; Yang et al., 2019; Radford et al., 2019; Lewis et al., 2020). However, they suffer from the limitation of lacking controllability in practical applications, for their generated texts may have toxic meanings or undesired topics. Hence how to control the text generation with desired attributes as topic, senti- ment, style, etc., has been further studied. To effectively control the desired attribute of the generated text, class-conditional language models (CC-LM) such as Conditional Transformer Lan- guage (CTRL) model (Keskar et al., 2019) gen- erate texts conditioned on control codes. But the controlled ability are restricted since the control ∗Corresponding author movie positive: In the end Hitchcock brings one of those rare Hollywood thrillers that I have been craving for a long time: the sus- pense thriller. It is one of my favorite genres and movies because you can always be sure that something exciting will happen in the end- ing (and sometimes it does!). I think it is also a classic Hitchcock masterpiece. automobile negative: In the past sixty years, GM cars have gotten worse, because GM en- gineers were worse. Now they are just as bad as Ford’s. But GM still makes good cars. I’m sorry but car makers need to go back to mak- ing things people want, like lawnmowers.It’s a big step backwards. Table 1: Examples of the text generated by our frame- work. The boldfaced words indicate the desired topic and sentiment. The underlined words are the prompts. The words related to the topic are highlighted in blue, and the words related to the sentiment are highlighted in red. codes are only used at the beginning of the genera- tion process. The Plug and Play Language Models (Dathathri et al., 2020) directly update the gradient of a …

Content truncated to 500 words