Controllable Paraphrase Generation with a Syntactic Exemplar

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5972–5984 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5972 Controllable Paraphrase Generation with a Syntactic Exemplar Mingda Chen Qingming Tang Sam Wiseman Kevin Gimpel Toyota Technological Institute at Chicago, Chicago, IL, 60637, USA {mchen,qmtang,swiseman,kgimpel}@ttic.edu Abstract Prior work on controllable text generation usu- ally assumes that the controlled attribute can take on one of a small set of values known a priori. In this work, we propose a novel task, where the syntax of a generated sen- tence is controlled rather by a sentential ex- emplar. To evaluate quantitatively with stan- dard metrics, we create a novel dataset with human annotations. We also develop a vari- ational model with a neural module specifi- cally designed for capturing syntactic knowl- edge and several multitask training objectives to promote disentangled representation learn- ing. Empirically, the proposed model is ob- served to achieve improvements over baselines and learn to capture desirable characteristics.1 1 Introduction Controllable text generation has recently become an area of intense focus in the natural language processing (NLP) community. Recent work has focused both on generating text satisfying certain stylistic requirements such as being formal or ex- hibiting a particular sentiment (Hu et al., 2017; Shen et al., 2017; Ficler and Goldberg, 2017), as well as on generating text meeting structural re- quirements, such as conforming to a particular template (Iyyer et al., 2018; Wiseman et al., 2018). These systems can be used in various applica- tion areas, such as text summarization (Fan et al., 2018), adversarial example generation (Iyyer et al., 2018), dialogue (Niu and Bansal, 2018), and data-to-document generation (Wiseman et al., 2018). However, prior work on controlled gen- eration has typically assumed a known, finite set of values that the controlled attribute can take on. In this work, we are interested instead in the novel setting where the generation is controlled 1Code and data are available at github.com/ mingdachen/syntactic-template-generation through an exemplar sentence (where any syntac- tically valid sentence is a valid exemplar). We will focus in particular on using a sentential exemplar to control the syntactic realization of a generated sentence. This task can benefit natural language interfaces to information systems by suggesting alternative invocation phrases for particular types of queries (Kumar et al., 2017). It can also bear on dialogue systems that seek to generate utterances that fit particular functional categories (Ke et al., 2018; Li et al., 2019). To address this task, we propose a deep gener- ative model with two latent variables, which are designed to capture semantics and syntax. To achieve better disentanglement between these two variables, we design multi-task learning objectives that make use of paraphrases and word order infor- mation. To further facilitate the learning of syntax, we additionally propose to train the syntactic com- ponent of our model with word noising and latent word-cluster codes. Word noising randomly re- places word tokens in the syntactic inputs based on a part-of-speech tagger …

Content truncated to 500 words