Pushing Paraphrase Away from Original Sentence: A Multi-Round Paraphrase Generation Approach

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1548–1557 August 1–6, 2021. ©2021 Association for Computational Linguistics 1548 Pushing Paraphrase Away from Original Sentence: A Multi-Round Paraphrase Generation Approach Zhe Lin and Xiaojun Wan Wangxuan Institute of Computer Technology, Peking University Center for Data Science, Peking University The MOE Key Laboratory of Computational Linguistics, Peking University {linzhe,wanxiaojun}@pku.edu.cn Abstract In recent years, neural paraphrase generation based on Seq2Seq has achieved superior per- formance, however, the generated paraphrase still has the problem of lack of diversity. In this paper, we focus on improving the diver- sity between the generated paraphrase and the original sentence, i.e., making generated para- phrase different from the original sentence as much as possible. We propose BTmPG (Back- Translation guided multi-round Paraphrase Generation), which leverages multi-round paraphrase generation to improve diversity and employs back-translation to preserve seman- tic information. We evaluate BTmPG on two benchmark datasets. Both automatic and hu- man evaluation show BTmPG can improve the diversity of paraphrase while preserving the se- mantics of the original sentence. 1 Introduction Paraphrase generation or sentence paraphrasing is an important task in natural language processing, and it requires rewriting a sentence while preserv- ing its semantics. Paraphrase generation has been widely used in many downstream tasks such as QA systems, semantic parsing, dialogue systems and so on. In recent years, deep learning techniques like sequence-to-sequence(Seq2Seq) have achieved su- perior performance on natural language genera- tion tasks (Zhao et al., 2010; Wubben et al., 2010). Many paraphrase models based on Seq2Seq have achieved inspiring results. For example, Prakash et al. (2016) leveraged stacked residual LSTM net- works to generate paraphrase, and Gupta et al. (2018) proposed a deep generative framework based on variational auto-encoder for paraphrase generation. Though paraphrase generation models based on Seq2Seq have demonstrated advanced ability, the generated paraphrase still has the problem of lack of diversity, i.e., the output paraphrase only makes trivial changes to the original sentence. A good paraphrase of a sentence is one that is semantically similar to that sentence while being (very) syntacti- cally and/or lexically different from it (Bhagat and Hovy, 2013). Paraphrase which is too similar to the original sentence is much less useful in many real applications. In this paper, we focus on improving the diver- sity of generated paraphrase, i.e., making generated paraphrase different from the original sentence as much as possible. An intuitive but uninvestigated idea is to adopt multi-round paraphrase generation. Concretely, we first send the original sentence into a paraphrase generation model to generate a para- phrase, and then we use the generated paraphrase as the input of the model to generate a new para- phrase. As long as we leverage a paraphrase gen- eration model with strong diversity like variational auto-encoder (VAE)(Kingma and Welling, 2013), we can get the paraphrase as different as possible from the original sentence after multi-round gener- ation. However, existing paraphrase models can not ensure that the major semantics of the original sentence can be preserved after multi-round para- …

Content truncated to 500 words