AESOP: Paraphrase Generation with Adaptive Syntactic Control

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5176–5189 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 5176 AESOP: Paraphrase Generation with Adaptive Syntactic Control Jiao Sun1,2, Xuezhe Ma1,2 and Nanyun Peng1,2,3 1Computer Science Department, University of Southern California 2Information Sciences Institute, University of Southern California 3Computer Science Department, University of California, Los Angeles jiaosun@usc.edu, xuezhema@usc.edu, violetpeng@cs.ucla.edu Abstract We propose to control paraphrase generation with carefully chosen target syntactic struc- tures to generate more proper and higher qual- ity paraphrases. Our model, AESOP, lever- ages a pretrained language model and pur- posefully selected syntactical control via a retrieval-based selection module to generate fluent paraphrases. Experiments show that AE- SOP achieves state-of-the-art performances on semantic preservation and syntactic conforma- tion on two benchmark datasets with ground- truth syntactic control from human-annotated exemplars. Moreover, with the retrieval-based target syntax selection module, AESOP gener- ates paraphrases with even better qualities than the current best model using human-annotated target syntactic parses according to human evaluation. We further demonstrate the effec- tiveness of AESOP to improve classification models’ robustness to syntactic perturbation by data augmentation on two GLUE tasks. 1 Introduction Syntactically-controlled paraphrase generation, which aims to generate paraphrases that conform with given syntactic structures, has drawn increas- ing attention in the community. On the one hand, paraphrase generation has benefited a wide range of NLP applications, such as neural machine trans- lation (Yang et al., 2019), dialogue generation (Gao et al., 2020), as well as improving model robust- ness (Huang et al., 2021) and interpretability (Jiang et al., 2019). On the other hand, syntactically- controlled paraphrasing has been used for diverse question generation (Yu and Jiang, 2021), diversi- fying creative generation (Tian et al., 2021) and improving model robustness (Iyyer et al., 2018; Huang and Chang, 2021). However, selecting suitable target syntactic struc- tures to control paraphrase generation for diverse and high-quality results is a lesser explored direc- tion. Prior works usually use a fixed set of syntactic AESOP What to say? How to guide paraphrasing? source parse (ROOT (S (NP ) (VP ) (. ) ) (ROOT (SBAR (IN ) (S ) (. )))� with the gold everything changes . the gold will change everything .� source sentence (ROOT (FRAG (PP ) (. ))) because of the gold , everything will change . (ROOT (S (NP ) (VP ) (. )))� all this gold 's gonna make things different. target parse paraphrase Figure 1: Given a source sentence, AESOP selects tar- get syntactic parses adaptively to guide paraphrase gen- eration. Paraphrases here are all generated by AESOP, which preserve the semantics from source sentences and conform with the selected syntactic parses. structures for all input sentences (Iyyer et al., 2018; Huang and Chang, 2021). A challenge with this method is that not all sentences can be paraphrased into the same set of syntactic structures. For exam- ple, it is impossible to turn a long sentence with multiple clauses into a noun phrase. Thus, Chen et al. (2019b) proposed …

Content truncated to 500 words