Best Student Forcing: A Simple Training Mechanism in Adversarial Language Generation

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 4680–4688 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 4680 Best Student Forcing: A Simple Training Mechanism in Adversarial Language Generation Jonathan Sauder*, Ting Hu*, Xiaoyin Che, Gonc¸alo Mordido, Haojin Yang, Christoph Meinel Hasso Plattner Institute, University of Potsdam Potsdam, Germany jonathan.sauder@student.hpi.de {ting.hu, xiaoyin.che, goncalo.mordido, haojin.yang, meinel.christoph}@hpi.de Abstract Language models trained with Maximum Likelihood Estimation (MLE) have been considered as a mainstream solution in Natural Language Generation (NLG) for years. Recently, various approaches with Generative Adversarial Nets (GANs) have also been proposed. While offering exciting new prospects, GANs in NLG by far are nevertheless reportedly suffering from training instability and mode collapse, and therefore outperformed by conventional MLE models. In this work, we propose techniques for improving GANs in NLG, namely Best Student Forcing (BSF), a novel yet simple adversarial training mechanism in which generated sequences of high quality are selected as temporary ground-truth to further train the generator. We also use an ensemble of discriminators to increase training stability and sample diversity. Evaluation shows that the combination of BSF and multiple discriminators consistently performs better than previous GAN approaches over various metrics, and outperforms a baseline MLE in terms of Fr´ech´et Distance, a recently proposed metric capturing both sample quality and diversity. Keywords: natural language generation, generative adversarial nets, reinforcement learning 1. Introduction Natural Language Generation (NLG) is a critical sub-task in many natural language processing tasks including ma- chine translation, image captioning, and conversational sys- tems. However, the generation of sequences that are seman- tically coherent and grammatically correct is difficult. Neu- ral networks trained with Maximum Likelihood Estimation (MLE) over Ground-Truth (GT) sentences have shown im- pressive results on many of these tasks (Karpathy and Fei- Fei, 2015; Wu et al., 2016; Hu et al., 2017), and been con- sidered as the mainstream solution in NLG. Recently, generative models of Variational AutoEncoder (VAE) are reportedly achieving inspiring results over M- LE (Schmidt, 2019). In VAE, a pair of encoder-decoder are trained simultaneously, while applies a latent loss typi- cally based on KL-divergence of the encoded latent vector from prior Gaussian distribution, in addition to reconstruc- tion loss calculated by MLE. After training, the decoder will do the generation job alone. Apparently, VAE models should also be trained only with GT sentences. Meanwhile, Generative Adversarial Nets (GANs) (Good- fellow et al., 2014) have been introduced into NLG (Yu et al., 2017; Che et al., 2017; Guo et al., 2017; Lin et al., 2017), bringing in new possibility to train the model further with guiding signals not directly from GT. However, vari- ous reports indicate that language GANs have shown short- comings in terms of training stability and sample diversi- ty, which make them less competitive in performance with conventional language models trained with MLE (Caccia et al., 2018; Semenuita et al., 2018; Tevet et al., 2018; Zhu et al., 2018), not to mention VAE. This fact motivates us to work …

Content truncated to 500 words