Word-level Textual Adversarial Attacking as Combinatorial Optimization

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6066–6080 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 6066 Word-level Textual Adversarial Attacking as Combinatorial Optimization Yuan Zang1∗, Fanchao Qi1∗, Chenghao Yang2∗†, Zhiyuan Liu1‡, Meng Zhang3, Qun Liu3, Maosong Sun1 1Department of Computer Science and Technology, Tsinghua University Institute for Artificial Intelligence, Tsinghua University Beijing National Research Center for Information Science and Technology 2Columbia University 3Huawei Noah’s Ark Lab {zangy17,qfc17}@mails.tsinghua.edu.cn, yangalan1996@gmail.com {liuzy,sms}@tsinghua.edu.cn, {zhangmeng92,qun.liu}@huawei.com Abstract Adversarial attacks are carried out to reveal the vulnerability of deep neural networks. Tex- tual adversarial attacking is challenging be- cause text is discrete and a small perturba- tion can bring significant change to the orig- inal input. Word-level attacking, which can be regarded as a combinatorial optimization problem, is a well-studied class of textual at- tack methods. However, existing word-level attack models are far from perfect, largely be- cause unsuitable search space reduction meth- ods and inefficient optimization algorithms are employed. In this paper, we propose a novel attack model, which incorporates the sememe- based word substitution method and particle swarm optimization-based search algorithm to solve the two problems separately. We con- duct exhaustive experiments to evaluate our at- tack model by attacking BiLSTM and BERT on three benchmark datasets. Experimental re- sults demonstrate that our model consistently achieves much higher attack success rates and crafts more high-quality adversarial examples as compared to baseline methods. Also, fur- ther experiments show our model has higher transferability and can bring more robustness enhancement to victim models by adversarial training. All the code and data of this paper can be obtained on https://github.com/ thunlp/SememePSO-Attack. 1 Introduction Adversarial attacks use adversarial examples (Szegedy et al., 2014; Goodfellow et al., 2015), which are maliciously crafted by perturbing the original input, to fool the deep neural networks ∗ Indicates equal contribution. Yuan developed the method, designed and conducted most experiments; Fanchao formalized the task, designed some experiments and wrote the paper; Chenghao made the original research proposal, per- formed human evaluation and conducted some experiments. †Work done during internship at Tsinghua University ‡Corresponding author FondOf produce shows Original Input Substitute Words Sememes movie cinema picture love like this film I Search Space Adversarial Example film this like I I this enjoy Figure 1: An example showing search space reduction with sememe-based word substitution and adversarial example search in word-level adversarial attacks. (DNNs). Extensive studies have demonstrated that DNNs are vulnerable to adversarial attacks, e.g., minor modification to highly poisonous phrases can easily deceive Google’s toxic comment detec- tion systems (Hosseini et al., 2017). From another perspective, adversarial attacks are also used to improve robustness and interpretability of DNNs (Wallace et al., 2019). In the field of natural lan- guage processing (NLP) which widely employs DNNs, practical systems such as spam filtering (Stringhini et al., 2010) and malware detection (Kolter and Maloof, 2006) have been broadly used, but at the same time the concerns about their secu- rity are growing. Therefore, the research on textual adversarial …

Content truncated to 500 words