Decomposable Neural Paraphrase Generation

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3403–3414 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3403 Decomposable Neural Paraphrase Generation Zichao Li, Xin Jiang, Lifeng Shang, Qun Liu Huawei Noah’s Ark Lab {li.zichao, jiang.xin, shang.lifeng, qun.liu}@huawei.com Abstract Paraphrasing exists at different granularity lev- els, such as lexical level, phrasal level and sen- tential level. This paper presents Decompos- able Neural Paraphrase Generator (DNPG), a Transformer-based model that can learn and generate paraphrases of a sentence at differ- ent levels of granularity in a disentangled way. Specifically, the model is composed of multiple encoders and decoders with differ- ent structures, each of which corresponds to a specific granularity. The empirical study shows that the decomposition mechanism of DNPG makes paraphrase generation more in- terpretable and controllable. Based on DNPG, we further develop an unsupervised domain adaptation method for paraphrase generation. Experimental results show that the proposed model achieves competitive in-domain perfor- mance compared to the state-of-the-art neural models, and significantly better performance when adapting to a new domain. 1 Introduction Paraphrases are texts that convey the same mean- ing using different wording. Paraphrase genera- tion is an important technique in natural language processing (NLP), which can be applied in various downstream tasks such as information retrieval, semantic parsing, and dialogue systems. Neu- ral sequence-to-sequence (Seq2Seq) models have demonstrated the superior performances on gener- ating paraphrases given a sentence (Prakash et al., 2016; Cao et al., 2017; Li et al., 2018; Ma et al., 2018). All of the existing works learn to para- phrase by mapping a sequence to another, with each word processed and generated in a uniform way. This work is motivated by a commonly ob- served phenomenon that the paraphrase of a sen- tence is usually composed of multiple paraphras- ing patterns at different levels of granularity, e.g., from the lexical to phrasal to sentential levels. For instance, the following pair of paraphrases con- tains both the phrase-level and the sentence-level patterns. what is the reason of $x → what makes $x happen world war II → the second world war Specifically, the blue part is the sentence-level pattern, which can be expressed as a pair of sentence templates, where $x can be any frag- ment of text. The green part is the phrase-level pattern, which is a pair of phrases. Table 1 shows more examples of paraphrase pairs sampled from WikiAnswers corpus 1 and Quora question pairs 2. We can see that the sentence-level para- phrases are more general and abstractive, while the word/phrase-level paraphrases are relatively diverse and domain-specific. Moreover, we no- tice that in many cases, paraphrasing can be de- coupled, i.e., the word-level and phrase-level pat- terns are mostly independent of the sentence-level paraphrase patterns. To address this phenomenon in paraphrase generation, we propose Decomposable Neural Paraphrase Generator (DNPG). Specifically, the DNPG consists of a separator, multiple encoders and decoders, and an aggregator. The sepa- rator first partitions an input sentence into seg- …

Content truncated to 500 words