Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLP

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11222 - 11237 December 7-11, 2022 ©2022 Association for Computational Linguistics Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLP WARNING: This paper contains real-world cases which are offensive in nature. Yangyi Chen1,2∗, Hongcheng Gao1,3∗, Ganqu Cui1, Fanchao Qi1 Longtao Huang4, Zhiyuan Liu1,5†, Maosong Sun1,5† 1NLP Group, DCST, IAI, BNRIST, Tsinghua University, Beijing 2University of Illinois Urbana-Champaign 3Chongqing University 4Alibaba Group 5 IICTUS, Shanghai yangyic3@illinois.edu, gaohongcheng2000@gmail.com Abstract Textual adversarial samples play important roles in multiple subfields of NLP research, including security, evaluation, explainability, and data augmentation. However, most work mixes all these roles, obscuring the problem definitions and research goals of the security role that aims to reveal the practical concerns of NLP models. In this paper, we rethink the re- search paradigm of textual adversarial samples in security scenarios. We discuss the deficien- cies in previous work and propose our sugges- tions that the research on the Security-oriented adversarial NLP (SoadNLP) should: (1) eval- uate their methods on security tasks to demon- strate the real-world concerns; (2) consider real- world attackers’ goals, instead of developing impractical methods. To this end, we first col- lect, process, and release a security datasets collection Advbench. Then, we reformalize the task and adjust the emphasis on different goals in SoadNLP. Next, we propose a simple method based on heuristic rules that can easily fulfill the actual adversarial goals to simulate real-world attack methods. We conduct experi- ments on both the attack and the defense sides on Advbench. Experimental results show that our method has higher practical value, indicat- ing that the research paradigm in SoadNLP may start from our new benchmark. All the code and data of Advbench can be obtained at https: //github.com/thunlp/Advbench. 1 Introduction Natural language processing (NLP) models based on deep learning have been employed in many real- world applications (Badjatiya et al., 2017; Zhang et al., 2018; Niklaus et al., 2018; Han et al., 2021). Meanwhile, there is a concurrent line of research on textual adversarial samples that are intentionally crafted to mislead models’ predictions (Samanta ∗Indicates equal contribution. Work done during intern- ship at Tsinghua University. †Corresponding Author. Role Explanation Security Adversarial samples can reveal the practical concerns of NLP models deployed in security situations. Evaluation Adversarial samples can be employed to benchmark models’ robustness to out-of-distribution data (diverse user inputs). Explainability Adversarial samples can explain part of the models’ decision processes. Augmentation Adversarial training based on adversarial samples augmenta- tion can improve performance and robustness. Table 1: Roles of textual adversarial samples. and Mehta, 2017; Papernot et al., 2016). Previous work shows that textual adversarial samples play important roles in multiple subfields of NLP re- search. We categorize and summarize the roles in Table 1. We argue that the problem definitions, including priorities of goals and experimental settings, are dif- ferent, considering the different roles of adversarial samples. However, most previous work in adversar- ial NLP mixes all different roles, …

Content truncated to 500 words