A Semantically Consistent and Syntactically Variational Encoder-Decoder Framework for Paraphrase Generation

Proceedings of the 28th International Conference on Computational Linguistics, pages 1186–1198 Barcelona, Spain (Online), December 8-13, 2020 1186 A Semantically Consistent and Syntactically Variational Encoder-Decoder Framework for Paraphrase Generation Wenqing Chen, Jidong Tian, Liqiang Xiao, Hao He∗, Yaohui Jin∗ MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University State Key Lab of Advanced Optical Communication System and Network, Shanghai Jiao Tong University {wenqingchen, hehao, jinyh}@sjtu.edu.cn Abstract Paraphrase generation aims to generate semantically consistent sentences with different syntac- tic realizations. Most of the recent studies rely on the typical encoder-decoder framework where the generation process is deterministic. However, in practice, the ability to generate multiple syntactically different paraphrases is important. Recent work proposed to cooperate variational inference on a target-related latent variable to introduce the diversity. But the latent variable may be contaminated by the semantic information of other unrelated sentences, and in turn, change the conveyed meaning of generated paraphrases. In this paper, we propose a semantically consis- tent and syntactically variational encoder-decoder framework, which uses adversarial learning to ensure the syntactic latent variable be semantic-free. Moreover, we adopt another discriminator to improve the word-level and sentence-level semantic consistency. So the proposed framework can generate multiple semantically consistent and syntactically different paraphrases. The exper- iments show that our model outperforms the baseline models on the metrics based on both n-gram matching and semantic similarity, and our model can generate multiple different paraphrases by assembling different syntactic variables. 1 Introduction Paraphrase generation is a longstanding problem in Natural Language Processing (NLP) (McKeown, 1983), which aims to generate semantically consistent sentences for a given sentence with different syntactic realizations. The task is not only an important building block for many text generation systems such as question answering (Buck et al., 2018; Dong et al., 2017), machine translation (Cho et al., 2014), but also beneficial to some NLP tasks such as semantic parsing (Su and Yan, 2017), sentence-level representation learning (Patro et al., 2018), data augmentation (Kumar et al., 2019). Neural network-based methods (Prakash et al., 2016; Gupta et al., 2018; Li et al., 2018; Fu et al., 2019) have shown great progress on paraphrase generation. The models mainly rely on the sequence-to- sequence (seq2seq) learning framework (Sutskever et al., 2014) with typical encoder-decoders, which are relatively deterministic during the testing stage. Generally, the models will select the best result through the beam search but are not able to produce multiple paraphrases in a principled way (Gupta et al., 2018). Due to the nature of beam-search, the quality of k-th variant will be worse than the first variant. In practice, the ability to generate multiple high-quality and diverse paraphrases is an important char- acteristic of text generation systems. A target-oriented seq2seq model is applaudable to achieve this goal. For example, Gupta et al. (2018) applied variational inference (Kingma and Welling, 2014) on a target-related latent variable z. During testing, the model can sample multiple latent variables z from a prior distribution to generate multiple different paraphrases. But the remained problem …

Content truncated to 500 words