Factorising Meaning and Form for Intent-Preserving Paraphrasing

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 1405–1418 August 1–6, 2021. ©2021 Association for Computational Linguistics 1405 Factorising Meaning and Form for Intent-Preserving Paraphrasing Tom Hosking Mirella Lapata Institute for Language, Cognition and Computation School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB tom.hosking@ed.ac.uk mlap@inf.ed.ac.uk Abstract We propose a method for generating para- phrases of English questions that retain the original intent but use a different surface form. Our model combines a careful choice of train- ing objective with a principled information bot- tleneck, to induce a latent encoding space that disentangles meaning and form. We train an encoder-decoder model to reconstruct a ques- tion from a paraphrase with the same meaning and an exemplar with the same surface form, leading to separated encoding spaces. We use a Vector-Quantized Variational Autoencoder to represent the surface form as a set of dis- crete latent variables, allowing us to use a clas- sifier to select a different surface form at test time. Crucially, our method does not require access to an external source of target exem- plars. Extensive experiments and a human evaluation show that we are able to generate paraphrases with a better tradeoff between se- mantic preservation and syntactic novelty com- pared to previous methods. 1 Introduction A paraphrase of an utterance is “an alternative sur- face form in the same language expressing the same semantic content as the original form” (Mad- nani and Dorr, 2010). For questions, a paraphrase should have the same intent, and should lead to the same answer as the original, as in the examples in Table 1. Question paraphrases are of significant interest, with applications in data augmentation (Iyyer et al., 2018), query rewriting (Dong et al., 2017) and duplicate question detection (Shah et al., 2018), as they allow a system to better identify the underlying intent of a user query. Recent approaches to paraphrasing use informa- tion bottlenecks with VAEs (Bowman et al., 2016) or pivot languages (Wieting and Gimpel, 2018) to try to extract the semantics of an input utterance, before projecting back to a (hopefully different) surface form. However, these methods have lit- How is a dialect different from a language? The differences between language and dialect? What is the difference between language and dialect? What is the weight of an average moose? Average weight of the moose? How much do moose weigh? How heavy is a moose? What country do parrots live in? In what country do parrots live? Where do parrots naturally live? What part of the world do parrots live in? Table 1: Examples of question paraphrase clusters, drawn from Paralex (Fader et al., 2013). Each mem- ber of the cluster has essentially the same semantic in- tent, but a different surface form. Each cluster exhibits variation in word choice, syntactic structure and even question type. Our task is to generate these different surface forms, using only a single example …

Content truncated to 500 words