Revisiting Pivot-Based Paraphrase Generation: Language Is Not the Only Optional Pivot

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4255–4268 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 4255 Revisiting Pivot-Based Paraphrase Generation: Language Is Not the Only Optional Pivot Yitao Cai∗, Yue Cao∗and Xiaojun Wan Wangxuan Institute of Computer Technology, Peking University Center for Data Science, Peking University The MOE Key Laboratory of Computational Linguistics, Peking University {caiyitao,yuecao,wanxiaojun}@pku.edu.cn Abstract Paraphrases refer to texts that convey the same meaning with different expression forms. Pivot-based methods, also known as the round- trip translation, have shown promising results in generating high-quality paraphrases. How- ever, existing pivot-based methods all rely on language as the pivot, where large-scale, high- quality parallel bilingual texts are required. In this paper, we explore the feasibility of using semantic and syntactic representations as the pivot for paraphrase generation. Concretely, we transform a sentence into a variety of differ- ent semantic or syntactic representations (in- cluding AMR, UD, and latent semantic repre- sentation), and then decode the sentence back from the semantic representations. We fur- ther explore a pretraining-based approach to compress the pipeline process into an end- to-end framework. We conduct experiments comparing different approaches with different kinds of pivots. Experimental results show that taking AMR as pivot can obtain paraphrases with better quality than taking language as the pivot. The end-to-end framework can re- duce semantic shift when language is used as the pivot. Besides, several unsupervised pivot-based methods can generate paraphrases with similar quality as the supervised encoder- decoder model, which indicates that parallel data of paraphrases may not be necessary for paraphrase generation. 1 Introduction Paraphrase generation is an important and challeng- ing task in the field of Natural Language Processing (NLP), which can be applied in a variety of appli- cations such as information retrieval (Yan et al., 2016), question answering (Fader et al., 2014; Yin et al., 2015), machine translation (Cho et al., 2014), and so on. 1The first two authors contributed equally to this pa- per. Codes are available at https://github.com/caoy1996/Pivot- paraphrase. Traditionally, paraphrase generation is usually implemented using ruled-based models (Fader et al., 2014; Zhao et al., 2009), lexicon-based meth- ods (Bolshakov and Gelbukh, 2004; Kauchak and Barzilay, 2006), grammar-based methods (Narayan et al., 2016), statistical machine translation-based methods (Quirk et al., 2004; Zhao et al., 2008). With the rapid development of deep learning tech- niques, neural methods have shown great power in paraphrase generation and achieve state-of-the-art results (Gupta et al., 2018; Yang et al., 2019a). Neu- ral paraphrase generation models usually follow the encoder-decoder paradigm. Given a sentence X, these models generate the paraphrase Y by directly modeling P(Y |X) through a deep neural network. However, deep neural networks are sensitive to do- mains in general (Stahlberg, 2020), while existing mainstream paraphrase corpora only cover a few specific domains, such as image caption (Lin et al., 2014) and questions (Fader et al., 2013). High- quality paraphrases for general domains are diffi- cult to obtain in practice, which greatly restricts the application of these seq2seq models. …

Content truncated to 500 words