Perturbations in the Wild: Leveraging Human-Written Text Perturbations for Realistic Adversarial Attack and Defense

Findings of the Association for Computational Linguistics: ACL 2022, pages 2953 - 2965 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Perturbations in the Wild: Leveraging Human-Written Text Perturbations for Realistic Adversarial Attack and Defense Thai Le Jooyoung Lee Kevin Yen∗ Yifan Hu∗ Dongwon Lee Penn State University {thaile, jfl5838, dongwon}@psu.edu Yahoo Research∗ {kevinyen, yifanhu}@yahooinc.com∗ DISCLAIMER! THIS PAPER CONTAINS EXAMPLE TEXTS THAT ARE OFFENSIVE IN NATURE Abstract We proposes a novel algorithm, ANTHRO, that inductively extracts over 600K human-written text perturbations in the wild and leverages them for realistic adversarial attack. Unlike existing character-based attacks which often deductively hypothesize a set of manipula- tion strategies, our work is grounded on ac- tual observations from real-world texts. We find that adversarial texts generated by AN- THRO achieve the best trade-off between (1) attack success rate, (2) semantic preservation of the original text, and (3) stealthiness–i.e. indistinguishable from human writings hence harder to be flagged as suspicious. Specifi- cally, our attacks accomplished around 83% and 91% attack success rates on BERT and RoBERTa, respectively. Moreover, it outper- formed the TextBugger baseline with an in- crease of 50% and 40% in terms of seman- tic preservation and stealthiness when evalu- ated by both layperson and professional hu- man workers. ANTHRO can further enhance a BERT classifier’s performance in under- standing different variations of human-written toxic texts via adversarial training when com- pared to the Perspective API. Source code will be published at github.com/lethaiq/ perturbations-in-the-wild. 1 Introduction Machine learning (ML) models trained to opti- mize only the prediction performance are often vulnerable to adversarial attacks (Papernot et al., 2016; Wang et al., 2019). In the text domain, espe- cially, a character-based adversarial attacker aims to fool a target ML model by generating an adver- sarial text x∗ from an original text x by manipu- lating characters of different words in x, such that some properties of x are preserved (Li et al., 2018; Eger et al., 2019; Gao et al., 2018). We character- ize strong and practical adversarial attacks as three criteria: (1) attack performance, as measured by the ability to flip a target model’s predictions, (2) Figure 1: ANTHRO (Bottom) extracts and uses human- written perturbations for adversarial attacks instead of proposing a specific set of manipulation rules (Top). semantic preservation, as measured by the ability to preserve the meaning of an original text, and (3) stealthiness, as measured by how unlikely it is de- tected as machine-manipulation and removed by defense systems or human examiners (Figure 1). While the first two criteria are natural derivation from adversarial literature (Papernot et al., 2016), stealthiness is also important to be a practical at- tack under a mass-manipulation scenario. In fact, adversarial text generation remains a challenging task under practical settings. Previously proposed character-based attacks fol- low a deductive approach where the researchers hypothesize a set of text manipulation strategies that exploit some vulnerabilities of textual ML models (Figure 1). Although these deductively de- rived techniques can demonstrate superior attack performance, there is no guarantee …

Content truncated to 500 words