Residue-Based Natural Language Adversarial Attack Detection

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3836 - 3848 July 10-15, 2022 ©2022 Association for Computational Linguistics Residue-Based Natural Language Adversarial Attack Detection Vyas Raina ALTA Institute, Cambridge University vr313@cam.ac.uk Mark Gales ALTA Institute, Cambridge University mjfg@cam.ac.uk Abstract Deep learning based systems are susceptible to adversarial attacks, where a small, imper- ceptible change at the input alters the model prediction. However, to date the majority of the approaches to detect these attacks have been designed for image processing systems. Many popular image adversarial detection ap- proaches are able to identify adversarial exam- ples from embedding feature spaces, whilst in the NLP domain existing state of the art de- tection approaches solely focus on input text features, without consideration of model em- bedding spaces. This work examines what dif- ferences result when porting these image de- signed strategies to Natural Language Process- ing (NLP) tasks - these detectors are found to not port over well. This is expected as NLP systems have a very different form of input: discrete and sequential in nature, rather than the continuous and fixed size inputs for images. As an equivalent model-focused NLP detection approach, this work proposes a simple sentence- embedding "residue" based detector to identify adversarial examples. On many tasks, it out- performs ported image domain detectors and recent state of the art NLP specific detectors 1. 1 Introduction In the last decade deep learning based models have demonstrated success in a wide range of applica- tion areas, including Natural Language Processing (NLP) (Vaswani et al., 2017) and object recogni- tion (He et al., 2015). These systems may be de- ployed in mission critical situations, where there is the requirement for a high level of robustness. How- ever, Szegedy et al. (2014) demonstrated that deep models have an inherent weakness: small perturba- tions in the input can yield significant, undesired, changes in the output from the model. These input perturbations were termed adversarial examples and their generation adversarial attacks. 1Code is available at: https://github.com/ rainavyas/NAACL-2022-Residue-Detector Adversarial attacks have been developed for sys- tems operating in various domains: image sys- tems (Serban et al., 2020; Biggio and Roli, 2017; Bhambri et al., 2019) and NLP systems (Lin et al., 2014; Samanta and Mehta, 2017; Rosenberg et al., 2017). The characteristics of the input can be very different between these application domains. Broadly, the nature of inputs can be described us- ing two key attributes: static (fixed length) vs se- quential and continuous vs discrete. Under this categorisation, image inputs are continuous and static, whilst NLP inputs are discrete and sequen- tial. This work argues that due to the fundamental differences in the input and resulting adversarial perturbations in the different domains, adversarial attack behaviour can vary significantly from one domain to another. Hence, the extensive research on exploring and understanding adversarial pertur- bation behaviour in the continuous, static world of image systems does not necessarily transfer well to the NLP tasks. …

Content truncated to 500 words