Multi-task Learning for Paraphrase Generation With Keyword and Part-of-Speech Reconstruction

Findings of the Association for Computational Linguistics: ACL 2022, pages 1234 - 1243 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Multi-task Learning for Paraphrase Generation With Keyword and Part-of-Speech Reconstruction Xuhang Xie Xuesong Lu∗ Bei Chen School of Data Science and Engineering East China Normal University {xhxie@stu,xslu@dase,beichen@stu}.ecnu.edu.cn Abstract Paraphrase generation using deep learning has been a research hotspot of natural language processing in the past few years. While pre- vious studies tackle the problem from differ- ent aspects, the essence of paraphrase gen- eration is to retain the key semantics of the source sentence and rewrite the rest of the con- tent. Inspired by this observation, we pro- pose a novel two-stage model, PGKPR, for paraphrase generation with keyword and part- of-speech reconstruction. The rationale is to capture simultaneously the possible keywords of a source sentence and the relations between them to facilitate the rewriting. In the first stage, we identify the possible keywords using a pre- diction attribution technique, where the words obtaining higher attribution scores are more likely to be the keywords. In the second stage, we train a transformer-based model via multi- task learning for paraphrase generation. The novel learning task is the reconstruction of the keywords and part-of-speech tags, respectively, from a perturbed sequence of the source sen- tence. The learned encodings are then decoded to generate the paraphrase. We conduct the experiments on two commonly-used datasets, and demonstrate the superior performance of PGKPR over comparative models on multiple evaluation metrics. 1 Introduction The task of paraphrase generation is to rephrase a given sentence by preserving its key seman- tics. While the problem was solved using rule- based approaches (McKeown, 1979; Meteer and Shaked, 1988) and traditional machine learning techniques (Quirk et al., 2004; Wubben et al., 2010), recent attentions have been shifted to devis- ing effective deep neural networks (Prakash et al., 2016; Gupta et al., 2018; Li et al., 2018), which gen- erally adopt the encoder-decoder framework. More ∗ Xuesong Lu is the corresponding author. recently, controllable paraphrase generation has been extensively investigated and offers the mecha- nisms to guide the generation process by providing a reference such as a syntactic template (Iyyer et al., 2018; Goyal and Durrett, 2020; Huang and Chang, 2021), a sentential exemplar (Chen et al., 2019; Su et al., 2021) and so on. SRC what are good workouts to lose belly fat ? POS [WDT] [VBP] [TO] [.] TGT what is the best way to lose belly fat ? POS [WP] [VBZ] [DT] [TO] [.] GNT what are some good exercises to get rid of belly fat ? POS [WDT] [VBP] [DT] [TO] [.] Table 1: A running example. Although the problem has been studied from dif- ferent aspects, the fundamental goal of paraphrase generation is to preserve the key semantics of a source sentence and rewrite the rest of the con- tent. Taking the paraphrase pair in Table 1 as a running example, the key semantics are entailed by the words “good workouts”, “lose belly fat” and …

Content truncated to 500 words