Exploring Diverse Expressions for Paraphrase Generation

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3173–3182, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 3173 Exploring Diverse Expressions for Paraphrase Generation Lihua Qian1, Lin Qiu1, Weinan Zhang1, Xin Jiang2, Yong Yu1 1Shanghai Jiao Tong University {qianlihua, lqiu, wnzhang, yyu}@apex.sjtu.edu.cn 2Noah’s Ark Lab, Huawei Technologies jiang.xin@huawei.com Abstract Paraphrasing plays an important role in vari- ous natural language processing (NLP) tasks, such as question answering, information re- trieval and sentence simplification. Recently, neural generative models have shown promis- ing results in paraphrase generation. Howev- er, prior work mainly focused on single para- phrase generation, while ignoring the fact that diversity is essential for enhancing generaliza- tion capability and robustness of downstream applications. Few works have been done to solve diverse paraphrase generation. In this paper, we propose a novel approach with t- wo discriminators and multiple generators to generate a variety of different paraphrases. A reinforcement learning algorithm is applied to train our model. Our experiments on two real- world datasets demonstrate that our model not only gains a significant increase in diversity but also improves generation quality over sev- eral state-of-the-art baselines. 1 Introduction Paraphrases refer to texts with identical meaning but expressed in different ways. Paraphrase gen- eration has a wide range of applications. For ex- ample, paraphrases can be used to extend the ref- erence texts for the automatic evaluation of text generation tasks, which make the evaluation more robust and accurate (Madnani and Dorr, 2010). In addition to increasing reference texts, paraphrases can also augment training data for text classifica- tion problems, especially for tasks lack of training data. Besides, paraphrases can be used as varia- tions of queries for information retrieval and ques- tion answering systems. Query paraphrasing can narrow the gap between system comprehension and user questions (Fader et al., 2014; Yin et al., 2015). And systems can obtain better results from multiple feedbacks for different query variants. Traditional paraphrase generation approaches leverage manual-crafted rules (McKeown, 1983; Hassan et al., 2007). In recent years, sequence to sequence learning techniques have achieved im- provements in a variety of NLP tasks, such as machine translation (Bahdanau et al., 2014; Cho et al., 2014), text summarization (Nallapati et al., 2016) and dialogue systems (Serban et al., 2017). Specifically, the paraphrase generation can also be formulated as a sequence to sequence prob- lem. Existing neural paraphrase generation works (Prakash et al., 2016; Gupta et al., 2018; Li et al., 2018) also demonstrate the effectiveness of deep learning method. Although prior paraphrase generation method- s mainly consider only one single output for each input, the nature of paraphrasing indicates that we can paraphrase one sentence into several different sentences. Therefore, we hope to generate a vari- ety of paraphrases while ensuring quality. How- ever, most of the previous work dedicated to im- proving the quality of top generated sentence, but did not conduct in-depth research on the diversi- ty of …

Content truncated to 500 words