Paraphrasing Revisited with Neural Machine Translation

Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 881–893, Valencia, Spain, April 3-7, 2017. c⃝2017 Association for Computational Linguistics Paraphrasing Revisited with Neural Machine Translation Jonathan Mallinson, Rico Sennrich and Mirella Lapata Institute for Language, Cognition and Computation School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB J.Mallinson@ed.ac.uk, {rsennric,mlap}@inf.ed.ac.uk Abstract Recognizing and generating paraphrases is an important component in many natural language processing applications. A well- established technique for automatically extracting paraphrases leverages bilin- gual corpora to find meaning-equivalent phrases in a single language by “pivot- ing” over a shared translation in another language. In this paper we revisit bilin- gual pivoting in the context of neural ma- chine translation and present a paraphras- ing model based purely on neural net- works. Our model represents paraphrases in a continuous space, estimates the de- gree of semantic relatedness between text segments of arbitrary length, or generates candidate paraphrases for any source in- put. Experimental results across tasks and datasets show that neural paraphrases out- perform those obtained with conventional phrase-based pivoting approaches. 1 Introduction Paraphrasing can be broadly described as the task of using an alternative surface form to express the same semantic content (Madnani and Dorr, 2010). Much of the appeal of paraphrasing stems from its potential application to a wider range of NLP problems. Examples include query and pat- tern expansion (Riezler et al., 2007), summariza- tion (Barzilay, 2003), question answering (Lin and Pantel, 2001), semantic parsing (Berant and Liang, 2014), semantic role labeling (Woodsend and La- pata, 2014), and machine translation (Callison- Burch et al., 2006). Most of the recent literature has focused on the automatic extraction of paraphrases from various different types of corpora consisting of parallel, non-parallel, and comparable texts. One of the most successful proposals uses bilingual parallel corpora to induce paraphrases based on techniques from phrase-based statistical machine translation (SMT, Koehn et al. (2003)). The intuition behind Bannard and Callison-Burch’s (2005) bilingual pivoting method is that two English strings e1 and e2 that translate to the same foreign string f can be assumed to have the same meaning. The method then pivots over f to extract ⟨e1,e2⟩ as a pair of paraphrases. Drawing inspiration from syntax- based SMT, several subsequent efforts (Callison- Burch, 2008; Ganitkevitch et al., 2011) extended this technique to syntactic paraphrases leading to the creation of PPDB (Ganitkevitch et al., 2013; Ganitkevitch and Callison-Burch, 2014), a large- scale paraphrase database containing over a billion of paraphrase pairs in 23 different languages. In this paper we revisit the bilingual pivoting approach from the perspective of neural machine translation, a new approach to machine transla- tion based purely on neural networks (Kalchbren- ner and Blunsom, 2013; Bahdanau et al., 2014; Sutskever et al., 2014; Luong et al., 2015). At its core, NMT uses a deep neural network trained end-to-end to maximize the conditional probabil- ity of a correct translation given a source sen- tence, using a bilingual corpus. …

Content truncated to 500 words