Semantic-Preserving Adversarial Code Comprehension

Proceedings of the 29th International Conference on Computational Linguistics, pages 3017–3028 October 12–17, 2022. 3017 Semantic-Preserving Adversarial Code Comprehension Yiyang Li1,2, Hongqiu Wu1,2 and Hai Zhao1,2,∗ 1 Department of Computer Science and Engineering, Shanghai Jiao Tong University 2 Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University {eric-lee,wuhongqiu}@sjtu.edu.cn,zhaohai@cs.sjtu.edu.cn Abstract Based on the tremendous success of pre-trained language models (PrLMs) for source code com- prehension tasks, current literature studies ei- ther ways to further improve the performance (generalization) of PrLMs, or their robustness against adversarial attacks. However, they have to compromise on the trade-off between the two aspects and none of them consider improving both sides in an effective and practical way. To fill this gap, we propose Semantic-Preserving Adversarial Code Embeddings (SPACE) to find the worst-case semantic-preserving attacks while forcing the model to predict the correct la- bels under these worst cases. Experiments and analysis demonstrate that SPACE can stay ro- bust against state-of-the-art attacks while boost- ing the performance of PrLMs for code1. 1 Introduction Inspired by the tremendous success of large pre- trained language models (PrLMs) in natural lan- guage understanding (NLU) field (Devlin et al., 2019; Liu et al., 2019; Yang et al., 2019; Clark et al., 2020), many attempts have been made to pre-train a language model for source code compre- hension, aiming at learning good code embeddings to help code-related downstream tasks like detect- ing vulnerable codes, searching codes with natu- ral language queries or answering questions about codes (Zhou et al., 2019; Husain et al., 2019; Liu and Wan, 2021; Kanade et al., 2020a; Buratti et al., 2020; Feng et al., 2020; Guo et al., 2021). Based on this, current literature focuses on either meth- ods to further improve the performance of PrLMs (Jain et al., 2021; Ding et al., 2021; Phan et al., 2021; Wu et al., 2021), or their robustness against ∗ Corresponding author. This paper was partially sup- ported by Key Projects of National Natural Science Founda- tion of China (U1836222 and 61733011). 1Our codes and data are available at https:// github.com/EricLee8/SPACE adversarial attacks (Zhang et al., 2020; Springer et al., 2020; Yang et al., 2022; Srikant et al., 2021). However, they have to compromise on the trade-off between the two aspects and none of them consider improving both sides in an effective and practical way (Bielik and Vechev, 2020; Yefet et al., 2020). For example, CodeBERT (Feng et al., 2020) and GraphCodeBERT (Guo et al., 2021) are two most widely used PrLMs for code and they have signif- icantly boosted the performance of source code comprehension models. Unfortunately, these pow- erful models are vulnerable to adversarial attacks that slight perturbations can make them produce wrong labels (Yang et al., 2022). To tackle this problem, previous works attempt to adopt adversar- ial training framework or data augmentation tricks to improve model robustness, yet they fail to main- tain accuracy on the origin clean test set (Bielik and Vechev, 2020; Yefet et al., 2020), or …

Content truncated to 500 words