Improving Adversarial Text Generation with n-Gram Matching

Improving Adversarial Text Generation with n-Gram Matching Shijie Li Faculty of Engineering and IT University of Technology Sydney Australia Shijie.Li@student.uts.edu.au Massimo Piccardi Faculty of Engineering and IT University of Technology Sydney Australia Massimo.Piccardi@uts.edu.au Abstract In the past few years, generative adversarial networks (GANs) have become increasingly important in natural language generation. How- ever, their performance seems to still have a sig- nificant margin for improvement. For this rea- son, in this paper we propose a new adversarial training method that tackles some of the limita- tions of GAN training in unconditioned gener- ation tasks. In addition to the commonly used reward signal from the discriminator, our ap- proach leverages another reward signal which is based on the occurrence of n-gram matches between the generated sentences and the train- ing corpus. Thanks to the inherent correlation of this reward signal with the commonly used evaluation metrics such as BLEU, our approach implicitly bridges the gap between the objec- tives used during training and inference. To circumvent the non-differentiability issues as- sociated with a discrete objective, our approach leverages the reinforcement learning policy gra- dient theorem. Our experimental results show that the model trained with mixed rewards from both n-gram matching and the discriminator has been able to outperform other GAN-based models in terms of BLEU score and quality- diversity trade-off at a parity of computational budget. 1 Introduction Neural language generation (NLG) is an important research area in natural language processing (NLP) because of its fundamental role in many other tasks such as machine translation (Wu et al., 2016), text summarization (See et al., 2017), and dialog systems (Bordes and Weston, 2017). Typically, these models are trained using an approach called “teacher forc- ing”, where the probability of ground-truth tokens conditional on previous ground-truth tokens is maxi- mized during training (Goyal et al., 2016). Although this approach has reported substantial success, it bi- ases the model toward ground-truth samples during training, but it has to rely on its own generated sam- ples during inference, often resulting in a mismatch between the two data distributions and the so-called “exposure bias” problem. Researchers have made sev- eral attempts to alleviate this issue, including sched- uled sampling (Bengio et al., 2015), adversarial train- ing (Yu et al., 2017), and, more recently, optimal transport objectives (Wang et al., 2020). Generative adversarial networks (GANs) (Good- fellow et al., 2014), initially conceived for realistic image generation, have attained a good degree of suc- cess also in a variety of NLP tasks. Yu et al. (2017) has been one of the earliest attempts to demonstrate the potential of GANs for NLG. Following that, a series of related works have shown considerable im- provements in terms of sample quality. However, some of these approaches rely on sophisticated archi- tecture design such as LeakGAN (Guo et al., 2018), while others like RelGAN (Nie et al., 2019) are based on enhanced model capacities. Therefore, direct com- parison of the performance between different models becomes difficult since the quality of the generated …

Content truncated to 500 words