Rethinking Stealthiness of Backdoor Attack against NLP Models

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 5543–5557 August 1–6, 2021. ©2021 Association for Computational Linguistics 5543 Rethinking Stealthiness of Backdoor Attack against NLP Models Wenkai Yang1, Yankai Lin2, Peng Li2, Jie Zhou2, Xu Sun1, 3∗ 1Center for Data Science, Peking University 2Pattern Recognition Center, WeChat AI, Tencent Inc., China 3MOE Key Laboratory of Computational Linguistics, School of EECS, Peking University wkyang@stu.pku.edu.cn xusun@pku.edu.cn {yankailin, patrickpli, withtomzhou}@tencent.com Abstract Recent researches have shown that large nat- ural language processing (NLP) models are vulnerable to a kind of security threat called the Backdoor Attack. Backdoor attacked mod- els can achieve good performance on clean test sets but perform badly on those input sen- tences injected with designed trigger words. In this work, we point out a potential prob- lem of current backdoor attacking research: its evaluation ignores the stealthiness of back- door attacks, and most of existing backdoor attacking methods are not stealthy either to system deployers or to system users. To address this issue, we first propose two ad- ditional stealthiness-based metrics to make the backdoor attacking evaluation more cred- ible. We further propose a novel word-based backdoor attacking method based on negative data augmentation and modifying word em- beddings, making an important step towards achieving stealthy backdoor attacking. Ex- periments on sentiment analysis and toxic de- tection tasks show that our method is much stealthier while maintaining pretty good at- tacking performance. Our code is available at https://github.com/lancopku/SOS. 1 Introduction Deep neural networks (DNNs) are widely used in various areas, such as computer vi- sion (CV) (Krizhevsky et al., 2012; He et al., 2016) and natural language processing (NLP) (Sutskever et al., 2014; Vaswani et al., 2017; Devlin et al., 2019; Yang et al., 2019; Liu et al., 2019), and have shown their great abilities in recent years. Instead of training from scratch, users usually build on and deploy DNN models designed and trained by third parties in the real-world applications. However, this common practice raises a serious concern that DNNs trained and provided by third parties can ∗Corresponding Author be already backdoor attacked to perform well on normal samples while behaving badly on samples with specific designed patterns. The model that is injected with a backdoor is called a backdoored model. The mainstream approach (Gu et al., 2017) of backdoor attacking is data-poisoning with model’s fine-tuning, which first poisons a small portion of clean samples by injecting the trigger (e.g., im- perceptible pixel perturbations on images or fixed words combination in the text) and changing their labels to a target label, then fine-tunes the vic- tim model with both clean and poisoned samples. In NLP, it could be divided into two main cate- gories: word-based methods (Garg et al., 2020; Kurita et al., 2020; Yang et al., 2021) that choose a rare word which hardly appears in the clean text as the backdoor trigger, or sentence-based meth- ods (Dai et al., 2019; Chen et …

Content truncated to 500 words