On the Transferability of Adversarial Attacks against Neural Text Classifier

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1612–1625 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 1612 On the Transferability of Adversarial Attacks against Neural Text Classifier Liping Yuan1,2, Xiaoqing Zheng∗1,2, Yi Zhou1,2 Cho-Jui Hsieh3, Kai-Wei Chang3 1School of Computer Science, Fudan University, Shanghai, China 2Shanghai Key Laboratory of Intelligent Information Processing 3Department of Computer Science, University of California, Los Angeles, USA {lpyuan19,zhengxq,yizhou17}@fudan.edu.cn {chohsieh,kwchang}@cs.ucla.edu Abstract Deep neural networks are vulnerable to adver- sarial attacks, where a small perturbation to an input alters the model prediction. In many cases, malicious inputs intentionally crafted for one model can fool another model. In this paper, we present the first study to systemat- ically investigate the transferability of adver- sarial examples for text classification models and explore how various factors, including net- work architecture, tokenization scheme, word embedding, and model capacity, affect the transferability of adversarial examples. Based on these studies, we propose a genetic algo- rithm to find an ensemble of models that can be used to induce adversarial examples to fool almost all existing models. Such adversarial examples reflect the defects of the learning pro- cess and the data bias in the training set. Fi- nally, we derive word replacement rules that can be used for model diagnostics from these adversarial examples. 1 Introduction Recent studies demonstrate that deep neural net- works are vulnerable to adversarial examples, inten- tionally crafted to fool the models. Although gen- erating adversarial examples for texts has shown to be more challenging than for images due to their discrete nature, many methods have been proposed to generate adversarial text examples and reveal the vulnerability of deep neural networks in nat- ural language processing (NLP) tasks, including reading comprehension (Jia and Liang, 2017), text classification (Samanta and Mehta, 2017; Wong, 2017; Liang et al., 2018; Alzantot et al., 2018; Yang et al., 2020), machine translation (Zhao et al., 2018; Ebrahimi et al., 2018; Cheng et al., 2020), dialogue systems (Cheng et al., 2019), and dependency pars- ing (Zheng et al., 2020). These methods perturb text examples by replacing, scrambling, and eras- ing characters, words or other language units to fool an NLP model. web → network excellent → splendid routine → everyday toilet → bathroom ... Senate Panel gives … Deal in Congress … Nortel Downsizes … … A pool of representative neural models 1 1 2 2 3 3 n n ... ... Transferability rate between any pair of models An optimal ensemble Highly transferable adversarial examples Adversarial word replacement rules (a) (b) (c) (d) Identify dataset biases Generate high- transferability adversarial examples Cause almost all existing neural modes to make mistakes MODEL Figure 1: Overview of the study. (a) Given a pool of representative neural models, we compute the adversar- ial transferability rate between any pair of models; (b) A genetic algorithm is used to find an optimal ensem- ble with the minimum number of members so that the adversarial examples crafted by attacking the ensem- ble can strongly transfer …

Content truncated to 500 words