Generating Syntactically Controlled Paraphrases without Using Annotated Parallel Pairs

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 1022–1033 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 1022 Generating Syntactically Controlled Paraphrases without Using Annotated Parallel Pairs Kuan-Hao Huang University of California, Los Angeles khhuang@cs.ucla.edu Kai-Wei Chang University of California, Los Angeles kwchang@cs.ucla.edu Abstract Paraphrase generation plays an essential role in natural language process (NLP), and it has many downstream applications. How- ever, training supervised paraphrase models re- quires many annotated paraphrase pairs, which are usually costly to obtain. On the other hand, the paraphrases generated by existing unsupervised approaches are usually syntacti- cally similar to the source sentences and are limited in diversity. In this paper, we demon- strate that it is possible to generate syntac- tically various paraphrases without the need for annotated paraphrase pairs. We propose Syntactically controlled Paraphrase Genera- tor (SynPG), an encoder-decoder based model that learns to disentangle the semantics and the syntax of a sentence from a collection of unan- notated texts. The disentanglement enables SynPG to control the syntax of output para- phrases by manipulating the embedding in the syntactic space. Extensive experiments using automatic metrics and human evaluation show that SynPG performs better syntactic control than unsupervised baselines, while the qual- ity of the generated paraphrases is competitive. We also demonstrate that the performance of SynPG is competitive or even better than su- pervised models when the unannotated data is large. Finally, we show that the syntactically controlled paraphrases generated by SynPG can be utilized for data augmentation to im- prove the robustness of NLP models. 1 Introduction Paraphrase generation (McKeown, 1983) is a long- lasting task in natural language processing (NLP) and has been greatly improved by recently devel- oped machine learning approaches and large data collections. Paraphrase generation demonstrates the potential of machines in semantic abstraction and sentence reorganization and has already been applied to many NLP downstream applications, Figure 1: Paraphrase generation with syntactic control. Given a source sentence and a target syntactic specifi- cation (either a full parse tree or top levels of a parse tree), the model is expected to generate a paraphrase with the syntax following the given specification. such as question answering (Yu et al., 2018), chat- bot engines (Yan et al., 2016), and sentence simpli- fication (Zhao et al., 2018). In recent years, various approaches have been proposed to train sequence-to-sequence (seq2seq) models on a large number of annotated paraphrase pairs (Prakash et al., 2016; Mallinson et al., 2017; Cao et al., 2017; Egonmwan and Chali, 2019). Some of them control the syntax of output sen- tences to improve the diversity of paraphrase gener- ation (Iyyer et al., 2018; Goyal and Durrett, 2020; Kumar et al., 2020). However, collecting annotated pairs is expensive and induces challenges for some languages and domains. On the contrary, unsuper- vised approaches build paraphrase models without using parallel corpora (Li et al., 2018; Roy and Grangier, 2019; Zhang et al., 2019). Most of them are based on the variational …

Content truncated to 500 words