Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2048–2058 June 6–11, 2021. ©2021 Association for Computational Linguistics 2048 Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models Wenkai Yang1, Lei Li2, Zhiyuan Zhang2, Xuancheng Ren2, Xu Sun1, 2∗, Bin He3 1Center for Data Science, Peking University 2MOE Key Laboratory of Computational Linguistics, School of EECS, Peking University 3Huawei Noah’s Ark Lab {wkyang,lilei}@stu.pku.edu.cn {zzy1210,renxc,xusun}@pku.edu.cn hebin.nlp@huawei.com Abstract Recent studies have revealed a security threat to natural language processing (NLP) mod- els, called the Backdoor Attack. Victim mod- els can maintain competitive performance on clean samples while behaving abnormally on samples with a specific trigger word inserted. Previous backdoor attacking methods usually assume that attackers have a certain degree of data knowledge, either the dataset which users would use or proxy datasets for a sim- ilar task, for implementing the data poison- ing procedure. However, in this paper, we find that it is possible to hack the model in a data-free way by modifying one sin- gle word embedding vector, with almost no accuracy sacrificed on clean samples. Ex- perimental results on sentiment analysis and sentence-pair classification tasks show that our method is more efficient and stealthier. We hope this work can raise the awareness of such a critical security risk hidden in the em- bedding layers of NLP models. Our code is available at https://github.com/ lancopku/Embedding-Poisoning. 1 Introduction Deep neural networks (DNNs) have achieved great success in various areas, including computer vi- sion (CV) (Krizhevsky et al., 2012; Goodfellow et al., 2014; He et al., 2016) and natural language processing (NLP) (Hochreiter and Schmidhuber, 1997; Sutskever et al., 2014; Vaswani et al., 2017; Devlin et al., 2019; Yang et al., 2019; Liu et al., 2019). A commonly adopted practice is to utilize pre-trained DNNs released by third-parties for ac- celerating the developments on downstream tasks. However, researchers have recently revealed that such a paradigm can lead to serious security risks since the publicly available pre-trained models can be backdoor attacked (Gu et al., 2017; Kurita et al., 2020), by which an attacker can manipulate the ∗Corresponding Author model to always classify special inputs as a pre- defined class while keeping the model’s perfor- mance on normal samples almost unaffected. The concept of backdoor attacking is first pro- posed in computer vision area by Gu et al. (2017). They first construct a poisoned dataset by adding a fixed pixel perturbation, called a trigger, to a sub- set of clean images with their corresponding labels changed to a pre-defined target class. Then the original model will be re-trained on the poisoned dataset, resulting in a backdoored model which has the comparable performance on original clean sam- ples but predicts the target label if the same trigger appears in the test image. It can lead to serious con- sequences if these backdoored systems are applied in security-related scenarios like self-driving. Similarly, by replacing the pixel perturbation …

Content truncated to 500 words