A Sweet Rabbit Hole by DARCY: Using Honeypots to Detect Universal Trigger’s Adversarial Attacks

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 3831–3844 August 1–6, 2021. ©2021 Association for Computational Linguistics 3831 A Sweet Rabbit Hole by DARCY: Using Honeypots to Detect Universal Trigger’s Adversarial Attacks Thai Le Penn State University thaile@psu.edu Noseong Park Yonsei University noseong@yonsei.ac.kr Dongwon Lee Penn State University dongwon@psu.edu Abstract The Universal Trigger (UniTrigger) is a recently-proposed powerful adversarial tex- tual attack method. Utilizing a learning-based mechanism, UniTrigger generates a fixed phrase that, when added to any benign inputs, can drop the prediction accuracy of a textual neural network (NN) model to near zero on a target class. To defend against this attack that can cause significant harm, in this paper, we borrow the “honeypot” concept from the cybersecurity community and propose DARCY, a honeypot-based defense frame- work against UniTrigger. DARCY greedily searches and injects multiple trapdoors into an NN model to “bait and catch” potential attacks. Through comprehensive experiments across four public datasets, we show that DARCY detects UniTrigger’s adversarial attacks with up to 99% TPR and less than 2% FPR in most cases, while maintaining the prediction accuracy (in F1) for clean inputs within a 1% margin. We also demonstrate that DARCY with multiple trapdoors is also robust to a diverse set of attack scenarios with attackers’ varying levels of knowledge and skills. We release the source code of DARCY at: https://github.com/lethaiq/ ACL2021-DARCY-HoneypotDefenseNLP. 1 Introduction Adversarial examples in NLP refer to carefully crafted texts that can fool predictive machine learn- ing (ML) models. Thus, malicious actors, i.e., attackers, can exploit such adversarial examples to force ML models to output desired predictions. There are several adversarial example generation algorithms, most of which perturb an original text at either character (e.g., (Li et al., 2018; Gao et al., 2018)), word (e.g., (Ebrahimi et al., 2018; Jin et al.; Wallace et al., 2019; Gao et al., 2018; Garg and Ramakrishnan, 2020), or sentence level (e.g., (Le et al., 2020; Gan and Ng; Cheng et al.)). Original: this movie is awesome Attack: zoning zoombie this movie is awesome Prediction: Positive −→ Negative Original: this movie is such a waste! Attack: charming this movie is such a waste! Prediction: Negative −→ Positive Table 1: Examples of the UniTrigger Attack Because most of the existing attack methods are instance-based search methods, i.e., searching an adversarial example for each specific input, they do not usually involve any learning mechanisms. A few learning-based algorithms, such as the Uni- versal Trigger (UniTrigger) (Wallace et al., 2019), MALCOM (Le et al., 2020), Seq2Sick (Cheng et al.) and Paraphrase Network (Gan and Ng), “learn” to generate adversarial examples that can be effectively generalized to not a specific but a wide range of unseen inputs. In general, learning-based attacks are more at- tractive to attackers for several reasons. First, they achieve high attack success rates. For example, UniTrigger can drop the prediction accuracy of an NN model to near zero just by appending a learned …

Content truncated to 500 words