Multi-target Backdoor Attacks for Code Pre-trained Models

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7236–7254 July 9-14, 2023 ©2023 Association for Computational Linguistics Multi-target Backdoor Attacks for Code Pre-trained Models Yanzhou Li1, Shangqing Liu1∗, Kangjie Chen1, Xiaofei Xie2, Tianwei Zhang1, and Yang Liu3,1 1Nanyang Technological University 2Singapore Management University, 3Zhejiang Sci-Tech University {yanzhou001, liu.shangqing, kangjie001, tianwei.zhang, yangliu}@ntu.edu.sg, xiaofei.xfxie@gmail.com Abstract Backdoor attacks for neural code models have gained considerable attention due to the ad- vancement of code intelligence. However, most existing works insert triggers into task-specific data for code-related downstream tasks, thereby limiting the scope of attacks. Moreover, the majority of attacks for pre-trained models are designed for understanding tasks. In this pa- per, we propose task-agnostic backdoor attacks for code pre-trained models. Our backdoored model is pre-trained with two learning strate- gies (i.e., Poisoned Seq2Seq learning and token representation learning) to support the multi- target attack of downstream code understanding and generation tasks. During the deployment phase, the implanted backdoors in the victim models can be activated by the designed trig- gers to achieve the targeted attack. We evaluate our approach on two code understanding tasks and three code generation tasks over seven datasets. Extensive experiments demonstrate that our approach can effectively and stealthily attack code-related downstream tasks. 1 Introduction Inspired by the great success of pre-trained models in natural languages (Devlin et al., 2019; Raffel et al., 2020; Brown et al., 2020), a large number of pre-trained models for programming languages are proposed (Feng et al., 2020; Guo et al.; Wang et al., 2021b; Ahmad et al., 2021). These works pre-train models on a large corpus of code-related data and then upload their pre-trained models to the public such as HuggingFace 1, TensorFlow Model Gar- den 2, and Model Zoo 3 to facilitate other users to achieve code-intelligent applications by fine-tuning on a task-specific dataset. However, it is precisely because these models are easily obtainable that they ∗Corresponding author 1https://huggingface.co 2https://github.com/tensorflow/models 3https://modelzoo.co are more susceptible to attack, such as backdoor attack (Gu et al., 2017). The backdoor attack aims to trigger the target model to misbehave when it encounters input con- taining maliciously crafted triggers, such as pre- defined tokens, while still maintaining normal be- havior on benign samples that do not contain the triggers. Existing works for backdoor attacks on neural code models (Ramakrishnan and Albargh- outhi, 2022; Sun et al., 2022; Yang et al., 2023) mainly insert a set of triggers to the task-specific dataset at the fine-tuning phase to implant the back- door and achieve the goal of the attack. For exam- ple, CodePoisoner (Li et al., 2022) proposed four poisoning strategies to design triggers for the task- specific dataset (i.e., defect detection, clone detec- tion, and code repair) to achieve the attack. Com- pared with this type of attack, the task-agnostic backdoor attacks on pre-trained code models are es- pecially security-critical as once these backdoored pre-trained models are fine-tuned and deployed, the potential vulnerabilities can be exploited for a large number of …

Content truncated to 500 words