Syntax-Guided Controlled Generation of Paraphrases

Syntax-Guided Controlled Generation of Paraphrases Ashutosh Kumar1 Kabir Ahuja2∗ Raghuram Vadapalli3∗ Partha Talukdar1 1Indian Institute of Science, Bangalore 2Microsoft Research, Bangalore 3Google, London ashutosh@iisc.ac.in, kabirahuja2431@gmail.com raghuram.4350@gmail.com, ppt@iisc.ac.in Abstract Given a sentence (e.g., ‘‘I like mangoes’’) and a constraint (e.g., sentiment flip), the goal of controlled text generation is to produce a sentence that adapts the input sentence to meet the requirements of the constraint (e.g., ‘‘I hate mangoes’’). Going beyond such simple constraints, recent work has started explor- ing the incorporation of complex syntactic- guidance as constraints in the task of controlled paraphrase generation. In these methods, syntactic-guidance is sourced from a separate exemplar sentence. However, this prior work has only utilized limited syntactic information available in the parse tree of the exemplar sentence. We address this limitation in the paper and propose Syntax Guided Controlled Paraphraser (SGCP), an end-to-end frame- work for syntactic paraphrase generation. We find that SGCP can generate syntax-conforming sentences while not compromising on rele- vance. We perform extensive automated and human evaluations over multiple real-world English language datasets to demonstrate the efficacy of SGCP over state-of-the-art base- lines. To drive future research, we have made SGCP’s source code available.1 1 Introduction Controlled text generation is the task of producing a sequence of coherent words based on given constraints. These constraints can range from simple attributes like tense, sentiment polarity, and word-reordering (Hu et al., 2017; Shen et al., 2017; Yang et al., 2018) to more complex syntactic information. For example, given a sentence ‘‘The movie is awful!’’ and a simple constraint like flip ∗ This research was conducted internship at Indian Institute of Science.during the authors 1https://github.com/malllabiisc/SGCP. sentiment to positive, a controlled text generator is expected to produce the sentence ‘‘The movie is fantastic!’’. These constraints are important in not only providing information about what to say but also how to say it. Without any constraint, the ubiquitous sequence-to-sequence neural models often tend to produce degenerate outputs and favor generic utterances (Vinyals and Le, 2015; Li etal., 2016). Although simple attributes are helpful in addressing what to say, they provide very little information about how to say it. Syntactic control over generation helps in filling this gap by providing that missing information. Incorporating complex syntactic information has shown promising results in neural machine translation (Stahlberg et al., 2016; Aharoni and Goldberg, 2017; Yang et al., 2019), data-to-text generation (Peng et al., 2019), abstractive text- summarization (Cao et al., 2018), and adversarial text generation (Iyyer et al., 2018). Additionally, recent work (Iyyer et al., 2018; Kumar et al., 2019) has shown that augmenting lexical and syntactical variations in the training set can help in building better performing and more robust models. In this paper, we focus on the task of syn- tactically controlled paraphrase generation, that is, given an input sentence and a syntactic exemplar, produce a sentence that conforms to the syntax of the exemplar while retaining the meaning of the original input sentence. While syntactically controlled generation of paraphrases finds applications in multiple domains …

Content truncated to 500 words