An End-to-End Generative Architecture for Paraphrase Generation

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3132–3142, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 3132 An End-to-End Generative Architecture for Paraphrase Generation Qian Yang1∗, Zhouyuan Huo2, Dinghan Shen1 Yong Cheng3, Wenlin Wang1, Guoyin Wang1, Lawrence Carin1 1 Duke University 2 University of Pittsburgh 3 Google AI qian.yang@duke.edu Abstract Generating high-quality paraphrases is a fun- damental yet challenging natural language processing task. Despite the effectiveness of previous work based on generative models, there remain problems with exposure bias in recurrent neural networks, and often a failure to generate realistic sentences. To overcome these challenges, we propose the first end- to-end conditional generative architecture for generating paraphrases via adversarial train- ing, which does not depend on extra linguis- tic information. Extensive experiments on four public datasets demonstrate the proposed method achieves state-of-the-art results, out- performing previous generative architectures on both automatic metrics (BLEU, METEOR, and TER) and human evaluations. 1 Introduction Paraphrases convey the same meaning as the orig- inal sentences or text, but with different expres- sions in the same language. Paraphrase genera- tion aims to synthesize paraphrases of a given sen- tence automatically. This is a fundamental natu- ral language processing (NLP) task, and it is im- portant for many downstream applications (Mad- nani and Dorr, 2010; Yang et al., 2016, 2017; Pas- sonneau et al., 2018). For example, paraphrases can help diversify the response of chatbot engines (Yan et al., 2016), strengthen question answering (Harabagiu and Hickl, 2006; Duboue and Chu- Carroll, 2006; Fader et al., 2014), augment relation extraction (Romano et al., 2006), and extend the coverage of semantic parsers (Berant and Liang, 2014). Generating accurate and diverse paraphrases automatically is still very challenging. Traditional methods (McKeown, 1983; Bolshakov and Gel- bukh, 2004; Zhao et al., 2008) exploit how linguis- ∗ Corresponding author tic knowledge can improve the quality of gener- ated paraphrases, including shallow linguistic fea- tures (Zhao et al., 2009), and syntactic and seman- tic information (Kozlowski et al., 2003; Ellsworth and Janin, 2007). However, they are often domain- specific and hard to scale, or yield inferior results. With the help of growing large data and neu- ral network models, recent studies have shown promising results. Three families of deep learn- ing architectures have been investigated with the goal of generating high-quality paraphrases. The first is to formulate paraphrase generation as a sequence-to-sequence problem, following experi- ence from machine translation (Bahdanau et al., 2014; Cheng et al., 2016). Prakash et al. (2016) proposes stacked residual long short-term mem- ory networks (LSTMs), while Cao et al. (2017) makes use of the gated recurrent unit (GRU) as the recurrent unit. Shortly afterwards, several works have focused on providing extra informa- tion to enhance the encoder-decoder model. Ma et al. (2018) adds distributed word representations, Huang et al. (2018) considers a paraphrased dic- tionary and Iyyer et al. (2018) utilizes a syntactic parser. Recently Wang et al. (2018) …

Content truncated to 500 words