Paraphrase Generation by Learning How to Edit from Samples

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6010–6021 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 6010 Paraphrase Generation by Learning How to Edit from Samples Amirhossein Kazemnejad†, Mohammadreza Salehi‡, Mahdieh Soleymani Baghshah‡ †Iran University of Science and Technology, ‡Sharif University of Technology a kazemnejad@comp.iust.ac.ir, mrezasalehi@ce.sharif.edu, soleymani@sharif.edu Abstract Neural sequence to sequence text generation has been proved to be a viable approach to paraphrase generation. Despite promising re- sults, paraphrases generated by these models mostly suffer from lack of quality and diver- sity. To address these problems, we propose a novel retrieval-based method for paraphrase generation. Our model first retrieves a para- phrase pair similar to the input sentence from a pre-defined index. With its novel editor mod- ule, the model then paraphrases the input se- quence by editing it using the extracted rela- tions between the retrieved pair of sentences. In order to have fine-grained control over the editing process, our model uses the newly in- troduced concept of Micro Edit Vectors. both extracts and exploits these vectors usingIt the attention mechanism in the Transformer ar- chitecture. Experimental results show the su- periority of our paraphrase generation method in terms of both automatic metrics, and human evaluation of relevance, grammaticality, and diversity of generated paraphrases. 1 Introduction Paraphrases are texts conveying the same mean- ing while using different words (Bhagat and Hovy, 2013). Paraphrase generation is an important task in Natural Language Processing (NLP) that has many applications in other down-stream tasks, such as text summarization, question answering, seman- tic parsing, and information retrieval (Cao et al., 2017; Fader et al., 2014; Berant and Liang, 2014). Early works on paraphrasing mostly investi- gated rule-based or statistical machine translation approaches to this task (Bannard and Callison- Burch, 2005). With the recent advances of neural sequence-to-sequence (Seq2Seq) framework in dif- ferent NLP tasks, especially in machine translation, an increasing amount of literature have also applied Retreiver Training Corpus Step 1: Find the most similar pair Edit Provider Edit Performer Step 2: Generate the paraphrase Editor How can I increase my presence of mind ? What is best way to increase presence of mind ? How can I overcome absence of mind ? what is the best way to overcome absence of mind ? Figure 1: An overview of the proposed model. This model retrieves the most similar paraphrase pair to the input x from the training corpus (Retriever), computes a set of edit vectors [M, z] based on the retrieved pair (Edit Provider), and applies these edits to the input se- quence x to generate its paraphrase (Edit Performer). Seq2Seq models to the task of paraphrase gener- ation (Prakash et al., 2016; Gupta et al., 2018; Li et al., 2018). Although the proposed Seq2Seq methods for paraphrase generation have shown promising re- sults, they are not yet as dominant as their coun- terparts used in neural machine translation. The main reason is that the available training data for paraphrasing is scarce and domain-specific …

Content truncated to 500 words