Reevaluating Adversarial Examples in Natural Language

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3829–3839 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 3829 Reevaluating Adversarial Examples in Natural Language John X. Morris⇤, Eli Lifland⇤, Jack Lanchantin, Yangfeng Ji, Yanjun Qi Department of Computer Science, University of Virginia {jm8wx, edl9cy, jjl5sw, yj3fs, yq2h}@virginia.edu Abstract State-of-the-art attacks on NLP models lack a shared definition of what constitutes a success- ful attack. These differences make the attacks difficult to compare and hindered the use of ad- versarial examples to understand and improve NLP models. We distill ideas from past work into a unified framework: a successful natural language adversarial example is a perturbation that fools the model and follows four proposed linguistic constraints. We categorize previous attacks based on these constraints. For each constraint, we suggest options for human and automatic evaluation methods. We use these methods to evaluate two state-of-the-art syn- onym substitution attacks. We find that pertur- bations often do not preserve semantics, and 38% introduce grammatical errors. Next, we conduct human studies to find a threshold for each evaluation method that aligns with human judgment. Human surveys reveal that to suc- cessfully preserve semantics, we need to sig- nificantly increase the minimum cosine simi- larities between the embeddings of swapped words and between the sentence encodings of original and perturbed sentences. With con- straints adjusted to better preserve semantics and grammaticality, the attack success rate drops by over 70 percentage points. 1 1 Introduction One way to evaluate the robustness of a machine learning model is to search for inputs that produce incorrect outputs. Inputs intentionally designed to fool deep learning models are referred to as ad- versarial examples (Goodfellow et al., 2017). Ad- versarial examples have successfully tricked deep neural networks for image classification: two im- ages that look exactly the same to a human receive ⇤* Equal contribution 1Our code and datasets are available here. ������������������������������������ ��������������������������� ������������ ����������� ������������������������������������ ������������������������������� ����������� ����������� Figure 1: An adversarial example generated by TFAD- JUSTED for BERT fine-tuned on the Rotten Tomatoes sentiment analysis dataset. Swapping a single word causes the prediction to change from positive to neg- ative. completely different predictions from the classifier (Goodfellow et al., 2014). While applicable in the image case, the idea of an indistinguishable change lacks a clear analog in text. Unlike images, two different sequences of text are never entirely indistinguishable. This raises the question: if indistinguishable perturbations are not possible, what are adversarial examples in text? The literature contains many potential answers to this question, proposing varying definitions for suc- cessful adversarial examples (Zhang et al., 2019). Even attacks with similar definitions of success often measure it in different ways. The lack of a consistent definition and standardized evaluation has hindered the use of adversarial examples to understand and improve NLP models. 2 Therefore, we propose a unified definition for successful adversarial examples in natural lan- guage: perturbations that both fool the model and fulfill a set of linguistic constraints. In Section 2, we …

Content truncated to 500 words