Paraphrase Generation as Unsupervised Machine Translation

Proceedings of the 29th International Conference on Computational Linguistics, pages 6379–6391 October 12–17, 2022. 6379 Paraphrase Generation as Unsupervised Machine Translation Xiaofei Sun1,2, Yufei Tian3, Yuxian Meng2, Nanyun Peng3 Fei Wu1,5, Jiwei Li1,2 and Chun Fan4 1Zhejiang University, 2Shannon.AI, 3University of California, Los Angeles 4Peng Cheng Laboratory, 4National Biomedical Imaging Center, Peking University 4 Computer Center, Peking University, 5Shanghai AI Laboratory 5Shanghai Institute for Advanced Study of Zhejiang University xiaofei_sun@zju.edu.cn, yufeit@ucla.edu, jiwei_li@shannonai.com Abstract In this paper, we propose a new paradigm for paraphrase generation by treating the task as unsupervised machine translation (UMT) based on the assumption that there must be pairs of sentences expressing the same meaning in a large-scale unlabeled monolingual corpus. The proposed paradigm first splits a large unlabeled corpus into multiple clusters, and trains multi- ple UMT models using pairs of these clusters. Then based on the paraphrase pairs produced by these UMT models, a unified surrogate model can be trained to serve as the final SEQ2SEQ model to generate paraphrases, which can be directly used for test in the unsupervised setup, or be finetuned on labeled datasets in the super- vised setup. The proposed method offers mer- its over machine-translation-based paraphrase generation methods, as it avoids reliance on bilingual sentence pairs. It also allows human intervene with the model so that more diverse paraphrases can be generated using different fil- tering criteria. Extensive experiments on exist- ing paraphrase dataset for both the supervised and unsupervised setups demonstrate the effec- tiveness the proposed paradigm. 1 Introduction The goal of paraphrase generation (Prakash et al., 2016a; Cao et al., 2016; Ma et al., 2018; Wang et al., 2018) is to generate a sentence semantically identical to a given input sentence but with varia- tions in lexicon or syntax. It has been applied to various downstream NLP tasks such as parsing (Be- rant and Liang, 2014), question answering (Dong et al., 2017), summarization (Barzilay, 2004) and machine translation (Callison-Burch et al., 2006). Building a strong paraphrase generation system usually requires massive amounts of high-quality annotated paraphrase pairs, but existing labeled datasets (Lin et al., 2014; Fader et al., 2013; Lan et al., 2017) are either of small sizes or restricted in narrow domains. To avoid such a heavy reliance on labeled datasets, recent works have explored unsu- pervised methods (Li et al., 2018b; Fu et al., 2019; Siddique et al., 2020) to generate paraphrase with- out annotated training data, among which the back- translation based model is an archetype (Mallinson et al., 2017; Sokolov and Filimonov, 2020). It borrows the idea of back-translation (BT) in ma- chine translation (Sennrich et al., 2016) where the model first translates a sentence s1 into another sentence s2 in a different language (e.g., En→Fr), and then translates s2 back to s1. In this way, the model is able to generate paraphrases by harness- ing bilingual datasets, removing the need for label paraphrase data. However, BT-based models for paraphrase gen- eration have the following severe issues: firstly, BT- based systems rely on external …

Content truncated to 500 words