Improving Paraphrase Detection with the Adversarial Paraphrasing Task

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 7106–7116 August 1–6, 2021. ©2021 Association for Computational Linguistics 7106 Improving Paraphrase Detection with the Adversarial Paraphrasing Task Animesh Nighojkar, John Licato Advancing Machine and Human Reasoning Lab Department of Computer Science and Engineering University of South Florida Tampa, FL, USA {anighojkar,licato}@usf.edu Abstract If two sentences have the same meaning, it should follow that they are equivalent in their inferential properties, i.e., each sen- tence should textually entail the other. How- ever, many paraphrase datasets currently in widespread use rely on a sense of paraphrase based on word overlap and syntax. Can we teach them instead to identify paraphrases in a way that draws on the inferential properties of the sentences, and is not over-reliant on lexical and syntactic similarities of a sentence pair? We apply the adversarial paradigm to this question, and introduce a new adversarial method of dataset creation for paraphrase iden- tification: the Adversarial Paraphrasing Task (APT), which asks participants to generate se- mantically equivalent (in the sense of mutually implicative) but lexically and syntactically dis- parate paraphrases. These sentence pairs can then be used both to test paraphrase identifi- cation models (which get barely random accu- racy) and then improve their performance. To accelerate dataset generation, we explore au- tomation of APT using T5, and show that the resulting dataset also improves accuracy. We discuss implications for paraphrase detection and release our dataset in the hope of making paraphrase detection models better able to de- tect sentence-level meaning equivalence. 1 Introduction Although there are many definitions of ‘paraphrase’ in the NLP literature, most maintain that two sen- tences that are paraphrases have the same mean- ing or contain the same information. Pang et al. (2003) define paraphrasing as “expressing the same information in multiple ways” and Bannard and Callison-Burch (2005) call paraphrases “alternative ways of conveying the same information.” Ganitke- vitch et al. (2013) write that “paraphrases are dif- fering textual realizations of the same meaning.” A definition that seems to sufficiently encompass the others is given by Bhagat and Hovy (2013): “para- phrases are sentences or phrases that use different wording to convey the same meaning.” However, even that definition is somewhat imprecise, as it lacks clarity on what it assumes ‘meaning’ means. If paraphrasing is a property that can hold be- tween sentence pairs,1 then it is reasonable to as- sume that sentences that are paraphrases must have equivalent meanings at the sentence level (rather than exclusively at the levels of individual word meanings or syntactic structures). Here a useful test is that recommended by inferential role se- mantics or inferentialism (Boghossian, 1994; Pere- grin, 2006), which suggests that the meaning of a statement s is grounded in its inferential properties: what one can infer from s and from what s can be inferred. Building on this concept from inferentialism, we assert that if two sentences have the same inferen- tial …

Content truncated to 500 words