DivGAN: Towards Diverse Paraphrase Generation via Diversified Generative Adversarial Network

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2411–2421 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 2411 DivGAN: Towards Diverse Paraphrase Generation via Diversified Generative Adversarial Network Yue Cao, Xiaojun Wan Wangxuan Institute of Computer Technology, Peking University Center for Data Science, Peking University The MOE Key Laboratory of Computational Linguistics, Peking University {yuecao,wanxiaojun}@pku.edu.cn Abstract Paraphrases refer to texts that convey the same meaning with different expression forms. Tra- ditional seq2seq-based models on paraphrase generation mainly focus on the fidelity while ignoring the diversity of outputs. In this pa- per, we propose a deep generative model to generate diverse paraphrases. We build our model based on the conditional generative ad- versarial network, and propose to incorporate a simple yet effective diversity loss term into the model in order to improve the diversity of outputs. The proposed diversity loss max- imizes the ratio of pairwise distance between the generated texts and their corresponding la- tent codes, forcing the generator to focus more on the latent codes and produce diverse sam- ples. Experimental results on benchmarks of paraphrase generation show that our proposed model can generate more diverse paraphrases compared with baselines. 1 Introduction The task of paraphrase generation refers to rewrit- ing a given sentence to a new paraphrase sentence, which requires that the generated sentence and in- put sentence are different in expression form, but have the same expressed meaning. Paraphrase gen- eration is a fundamental task of natural language processing (NLP). The technique of paraphrase generation has been widely used in many down- stream applications, such as information retrieval, question answering, machine translation, and so on. Early works on paraphrase generation mainly focus on rule-based (McKeown, 1983; Meteer and Shaked, 1988), grammar-based (Narayan et al., 2016), lexicon-based (Bolshakov and Gelbukh, 2004; Kauchak and Barzilay, 2006), and statis- tical machine translation (SMT)-based methods (Kauchak and Barzilay, 2006; Zhao et al., 2009). Recently, with the release of large-scale paraphrase datasets, sequence-to-sequence (seq2seq) models (Prakash et al., 2016; Li et al., 2019; Kajiwara, 2019; Li et al., 2018; Gupta et al., 2018; Shakeri and Sethy, 2019; Yang et al., 2019) have become the dominant technique in the field of paraphrase generation. Paraphrases should be diversified in nature, i.e., an input sentence can correspond to multiple plau- sible paraphrases. Traditional seq2seq-based meth- ods tend to generate highly similar outputs since the maximum likelihood estimation (MLE)-based objective function mostly cares about the validity rather than the diversity of outputs. Some works introduce control mechanisms over seq2seq mod- els to produce diverse outputs (Iyyer et al., 2018; Park et al., 2019; Chen et al., 2019). However, the templates or exemplars in control mechanism can- not cover all the possibility of paraphrase, and the introduction of control mechanism is inflexible. Xu et al. (2018b) propose to use a shared decoder with different decoder embeddings to generate different outputs, but the decoder embeddings are not explic- itly encouraged and learned to produce different outputs. Generative models, such as Variational Autoen- coder (VAE) …

Content truncated to 500 words