Adversarial Semantic Collisions

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 4198–4210, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 4198 Adversarial Semantic Collisions Congzheng Song Cornell University cs2296@cornell.edu Alexander M. Rush Cornell Tech arush@cornell.edu Vitaly Shmatikov Cornell Tech shmat@cs.cornell.edu Abstract We study semantic collisions: texts that are semantically unrelated but judged as similar by NLP models. We develop gradient-based approaches for generating semantic collisions and demonstrate that state-of-the-art models for many tasks which rely on analyzing the meaning and similarity of texts—including paraphrase identification, document retrieval, response suggestion, and extractive summa- rization—are vulnerable to semantic colli- sions. For example, given a target query, insert- ing a crafted collision into an irrelevant doc- ument can shift its retrieval rank from 1000 to top 3. We show how to generate semantic collisions that evade perplexity-based filtering and discuss other potential mitigations. Our code is available at https://github.com/ csong27/collision-bert. 1 Introduction Deep neural networks are vulnerable to adversar- ial examples (Szegedy et al., 2014; Goodfellow et al., 2015), i.e., imperceptibly perturbed inputs that cause models to make wrong predictions. Ad- versarial examples based on inserting or modifying characters and words have been demonstrated for text classification (Liang et al., 2018; Ebrahimi et al., 2018; Pal and Tople, 2020), question answer- ing (Jia and Liang, 2017; Wallace et al., 2019), and machine translation (Belinkov and Bisk, 2018; Wal- lace et al., 2020). These attacks aim to minimally perturb the input so as it to preserve its semantics while changing the output of the model. In this work, we introduce and study a different class of vulnerabilities in NLP models for analyz- ing the meaning and similarity of texts. Given an input (query), we demonstrate how to generate a semantic collision: an unrelated text that is judged semantically equivalent by the target model. Se- mantic collisions are the “inverse” of adversarial examples. Whereas adversarial examples are sim- ilar inputs that produce dissimilar model outputs, semantic collisions are dissimilar inputs that pro- duce similar model outputs. We develop gradient-based approaches for gener- ating collisions given white-box access to a model and deploy them against several NLP tasks. For paraphrase identification, the adversary crafts col- lisions that are judged as a valid paraphrase of the input query; downstream applications such as re- moving duplicates or merging similar content will thus erroneously merge the adversary’s inputs with the victim’s inputs. For document retrieval, the ad- versary inserts collisions into one of the documents that cause it to be ranked very high even though it is irrelevant to the query. For response sugges- tion, the adversary’s irrelevant text is ranked as the top suggestion and can also carry spam or advertis- ing. For extractive summarization, the adversary inserts a collision into the input text, causing it to be picked as the most relevant content. Our first technique generates collisions aggres- sively, without regard to potential defenses. We then develop two techniques, “regularized aggres- sive” and “natural,” that constrain generated col- lisions using a language model so …

Content truncated to 500 words