CoCGAN: Contrastive Learning for Adversarial Category Text Generation

Proceedings of the 29th International Conference on Computational Linguistics, pages 6403–6414 October 12–17, 2022. 6403 CoCGAN: Contrastive Learning for Adversarial Category Text Generation Xin Sheng1,3, Linli Xu1,3∗, Yinlong Xu2, Changcun Bao4, Huang Chen4, Bo Ren4 1 Anhui Province Key Laboratory of Big Data Analysis and Application, School of Computer Science and Technology, University of Science and Technology of China. 2 School of Computer Science and Technology, University of Science and Technology of China. 3 State Key Laboratory of Cognitive Intelligence. 4 Tencent Youtu Lab. xins@mail.ustc.edu.cn, {linlixu,ylxu}@ustc.edu.cn, {changcunbao,huaangchen,timren}@tencent.com Abstract The task of generating texts of different cat- egories has attracted more and more atten- tion in the area of natural language generation recently. Meanwhile, generative adversarial net (GAN) has demonstrated its effectiveness on text generation, and is further applied to category text generation in later works. Dif- ferent from existing methods, which mainly consider the pairwise relations between the text embedding and the corresponding fixed one-hot class label (data-to-class relations), this paper proposes a novel Contrastive Category Gener- ative Adversarial Net (CoCGAN) to incorpo- rate contrastive learning into adversarial cate- gory text generation, considering more flexible data-to-class relations as well as relations be- tween the multiple text embeddings in the same batch (data-to-data relations). The discrimina- tor of CoCGAN discriminates the authenticity of given samples and optimizes a contrastive learning objective to capture both more flexi- ble data-to-class relations and data-to-data re- lations among training samples. Accordingly, the generator tries to produce more realistic samples which can confuse the discriminator. Experimental results on both synthetic and real category text generation datasets demonstrate that CoCGAN can achieve significant improve- ments over the baseline category text genera- tion models. 1 Introduction Category text generation is the task of generat- ing coherent and meaningful text with different categories and has received increasing attention in many natural language processing applications, such as sentiment analysis (Li et al., 2018) and di- alogue generation (Li et al., 2017). It is a further expression of machine intelligence, and makes the generated texts more friendly to humans. Category text generation is a generalization of sentimental ∗Corresponding author text generation and can be seen as a type of condi- tional text generation. This task focuses on how to integrate the auxiliary category information with the conventional text generation frameworks. Re- cently, generative adversarial net (GAN) (Good- fellow et al., 2014), where the discriminator is de- signed to guide the generator, is combined with the reinforcement learning (RL) (Williams, 1992) methods to generate discrete text sequences for general text generation with some remarkable suc- cesses (Yu et al., 2017; Guo et al., 2018; Caccia et al., 2020). Different from the general text gener- ation task, category text generation mainly focuses on automatically generating a variety of control- lable texts according to the specified categories. However, it is challenging to incorporate the cate- gory information into the sentences and design an appropriate objective for generating texts of differ- ent categories simultaneously. In previous works, attempts have been made to extend the general …

Content truncated to 500 words