PARSE: An Efficient Search Method for Black-box Adversarial Text Attacks

Proceedings of the 29th International Conference on Computational Linguistics, pages 4776–4787 October 12–17, 2022. 4776 PARSE: An Efficient Search Method for Black-box Adversarial Text Attacks Pengwei Zhan§‡, Chao Zheng§, Jing Yang§∗, Yuxiang Wang§, Liming Wang§, Yang Wu§‡, Yunjian Zhang§‡ §Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China ‡School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China {zhanpengwei,zhengchao1135,yangjing,wangyuxiang}@iie.ac.cn {wangliming,wuyang0419,zhangyunjian}@iie.ac.cn Abstract Neural networks are vulnerable to adversar- ial examples. The adversary can success- fully attack a model even without knowing model architecture and parameters, i.e., un- der a black-box scenario. Previous works on word-level attacks widely use word impor- tance ranking (WIR) methods and complex search methods, including greedy search and heuristic algorithms, to find optimal substitu- tions. However, these methods fail to balance the attack success rate and the cost of attacks, such as the number of queries to the model and the time consumption. In this paper, We propose PAthological woRd Saliency sEarch (PARSE) that performs the search under dy- namic search space following the subarea im- portance. Experiments show that PARSE can achieve comparable attack success rates to complex search methods while saving numer- ous queries and time, e.g., saving at most 74% of queries and 90% of time compared with greedy search when attacking the exam- ples from Yelp dataset. The adversarial exam- ples crafted by PARSE are also of high qual- ity, highly transferable, and can effectively im- prove model robustness in adversarial training. 1 Introduction Neural networks have achieved remarkable suc- cess in various NLP tasks while being vulnerable to adversarial examples. The adversary can craft adversarial examples, which contain noise that is imperceptible to human but can mislead the model decision, even without knowing the model archi- tecture and parameters. Under such black-box sce- nario, word-level attacks have been more focused on by recent studies for the flexibility of the attack and the high quality generated examples (Gao et al., 2018; Alzantot et al., 2018; Jin et al., 2020; Li et al., 2018; Garg and Ramakrishnan, 2020a; Ebrahimi et al., 2018). Word-level attacks can flexibly fit ∗Corresponding Author. Greedy Beam(w=4) Beam(w=8) Genetic PSO PARSE(w=2) PARSE(w=4) PARSE(w=8) TIWO Random WIR-UNK WIR-A WIR-Delete PARSE(w=1) Search Method A.S% #Queries Time(s) Random 83.02 252 3503 TIWO 82.31 191 1802 WIR-Delete 91.64 234 1289 WIR-A 91.82 225 1347 WIR-UNK 91.79 221 1325 PARSE (w = 1) 93.23 378 1568 PARSE (w = 2) 93.38 472 2031 PARSE (w = 4) 94.14 620 4670 PARSE (w = 8) 96.80 1065 5665 Greedy 96.38 3327 27494 Beam (w = 4) 96.99 10 159 85087 Beam (w = 8) 97.84 18 327 211 400 Genetic 97.13 13 469 136 602 PSO 98.51 51 341 402 452 Figure 1: Average #Queries vs. Attack success rate (%) when attacking TextCNN on 500 examples from Yelp in HowNet. Increasing the beam width w in PARSE effectively increases the attack success rate while just costing a few more #queries and time. PARSE (w=8) outperforms Greedy search while taking only 32% of queries …

Content truncated to 500 words