Contrasting Human- and Machine-Generated Word-Level Adversarial Examples for Text Classification

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8258–8270 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 8258 Contrasting Human- and Machine-Generated Word-Level Adversarial Examples for Text Classification Maximilian Mozes1 Max Bartolo1 Pontus Stenetorp1 Bennett Kleinberg2,1 Lewis D. Griffin1 1University College London 2Tilburg University {m.mozes, m.bartolo, p.stenetorp, l.griffin}@cs.ucl.ac.uk bennett.kleinberg@tilburguniversity.edu Abstract Research shows that natural language process- ing models are generally considered to be vul- nerable to adversarial attacks; but recent work has drawn attention to the issue of validating these adversarial inputs against certain criteria (e.g., the preservation of semantics and gram- maticality). Enforcing constraints to uphold such criteria may render attacks unsuccessful, raising the question of whether valid attacks are actually feasible. In this work, we in- vestigate this through the lens of human lan- guage ability. We report on crowdsourcing studies in which we task humans with itera- tively modifying words in an input text, while receiving immediate model feedback, with the aim of causing a sentiment classification model to misclassify the example. Our find- ings suggest that humans are capable of gen- erating a substantial amount of adversarial ex- amples using semantics-preserving word sub- stitutions. We analyze how human-generated adversarial examples compare to the recently proposed TEXTFOOLER, GENETIC, BAE and SEMEMEPSO attack algorithms on the dimen- sions naturalness, preservation of sentiment, grammaticality and substitution rate. Our find- ings suggest that human-generated adversarial examples are not more able than the best algo- rithms to generate natural-reading, sentiment- preserving examples, though they do so by be- ing much more computationally efficient. 1 Introduction The vulnerability of natural language processing (NLP) models to adversarial examples has received widespread attention (Alzantot et al., 2018; Iyyer et al., 2018; Ren et al., 2019). Text processing models have been shown to be susceptible to ad- versarial input perturbations across tasks, includ- ing question answering and text classification (Jia and Liang, 2017; Jin et al., 2019). The concept of adversarial examples originated in computer vi- sion (Szegedy et al., 2013; Goodfellow et al., 2014), and in that domain defines perturbations of input data to neural networks that are barely perceptible to the human viewer. Due to the discrete nature of text, however, that definition is less applicable in an NLP context, since every perturbation to the input tokens is unavoidably perceptible. Consequently, recent work aims to perturb textual inputs while preserving the sequence’s naturalness and seman- tics (i.e., rendering changes imperceptible on these dimensions). However, as shown by Morris et al. (2020a), achieving these desiderata is challenging because even small perturbations can render a text meaningless, grammatically incorrect or unnatu- ral, and furthermore several proposed adversarial attacks fail routinely to achieve them. If the algo- rithms are modified to ensure that they do achieve the desiderata then their rate of generating success- ful examples greatly diminishes, suggesting that the reported success rates of recently proposed at- tacks might represent an overestimation of their true capabilities. This, in turn, raises the question of whether valid word-level adversarial examples are routinely …

Content truncated to 500 words