ONION: A Simple and Effective Defense Against Textual Backdoor Attacks

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9558–9566 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 9558 ONION: A Simple and Effective Defense Against Textual Backdoor Attacks Fanchao Qi1,2∗, Yangyi Chen2,4∗†, Mukai Li2,5†, Yuan Yao1,2, Zhiyuan Liu1,2,3, Maosong Sun1,2,3‡ 1Department of Computer Science and Technology, Tsinghua University, Beijing, China 2Beijing National Research Center for Information Science and Technology 3Institute for Artificial Intelligence, Tsinghua University, Beijing, China 4Huazhong University of Science and Technology 5Beihang University qfc17@mails.tsinghua.edu.cn, yangyichen6666@gmail.com Abstract Backdoor attacks are a kind of emergent training-time threat to deep neural networks (DNNs). They can manipulate the output of DNNs and possess high insidiousness. In the field of natural language processing, some at- tack methods have been proposed and achieve very high attack success rates on multiple pop- ular models. Nevertheless, there are few stud- ies on defending against textual backdoor at- tacks. In this paper, we propose a simple and effective textual backdoor defense named ONION, which is based on outlier word de- tection and, to the best of our knowledge, is the first method that can handle all the tex- tual backdoor attack situations. Experiments demonstrate the effectiveness of our model in defending BiLSTM and BERT against five dif- ferent backdoor attacks. All the code and data of this paper can be obtained at https: //github.com/thunlp/ONION. 1 Introduction In recent years, deep neural networks (DNNs) have been deployed in various real-world appli- cations because of their powerful performance. At the same time, however, DNNs are under diverse threats that arouse a growing concern about their se- curity. Backdoor attacks (Gu et al., 2017), or trojan attacks (Liu et al., 2018b), are a kind of emergent in- sidious security threat to DNNs. Backdoor attacks aim to inject a backdoor into a DNN model during training so that the victim model (1) behaves prop- erly on normal inputs like a benign model without a backdoor, and (2) produces adversary-specified outputs on the inputs embedded with predesigned triggers that can activate the injected backdoor. Backdoor attacks are very stealthy, because a backdoored model is almost indistinguishable from a benign model unless receiving trigger-embedded ∗Equal contribuction †Work done during internship at Tsinghua University ‡Corresponding author. Email: sms@tsinghua.edu.cn inputs. Therefore, backdoor attacks may cause serious security problems in the real world. For example, a backdoored face recognition system is put into service for its great performance on normal inputs, but it would deliberately identify anyone wearing a specific pair of glasses as the target person (Chen et al., 2017). Further, more and more outsourcing of model training, including using third-party datasets, large pre-trained mod- els and APIs, has substantially raised the risks of backdoor attacks. In short, the threat of backdoor attacks is increasingly significant. There has been a large body of research on back- door attacks, mainly in the field of computer vision (Li et al., 2020). The most common attack method is training data poisoning, which injects a back- door into a victim model by training the …

Content truncated to 500 words