“That Is a Suspicious Reaction!”: Interpreting Logits Variation to Detect NLP Adversarial Attacks

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7806 - 7816 May 22-27, 2022 c⃝2022 Association for Computational Linguistics "That Is a Suspicious Reaction!": Interpreting Logits Variation to Detect NLP Adversarial Attacks Edoardo Mosca TU Munich, Department of Informatics, Germany edoardo.mosca@tum.de Shreyash Agarwal TU Munich, Department of Informatics, Germany shreyash.agarwal@tum.de Javier Rando-Ramirez ETH Zurich, Department of Computer Science, Switzerland jrando@student.ethz.ch Georg Groh TU Munich, Department of Informatics, Germany grohg@in.tum.de Abstract Adversarial attacks are a major challenge faced by current machine learning research. These purposely crafted inputs fool even the most advanced models, precluding their de- ployment in safety-critical applications. Ex- tensive research in computer vision has been carried to develop reliable defense strategies. However, the same issue remains less explored in natural language processing. Our work presents a model-agnostic detector of adver- sarial text examples. The approach identi- fies patterns in the logits of the target classi- fier when perturbing the input text. The pro- posed detector improves the current state-of- the-art performance in recognizing adversarial inputs and exhibits strong generalization capa- bilities across different NLP models, datasets, and word-level attacks. 1 Introduction Despite recent advancements in Natural Language Processing (NLP), adversarial text attacks continue to be highly effective at fooling models into mak- ing incorrect predictions (Ren et al., 2019; Wang et al., 2019; Garg and Ramakrishnan, 2020). In par- ticular, syntactically and grammatically consistent attacks are a major challenge for current research as they do not alter the semantical information and are not detectable via spell checkers (Wang et al., 2019). While some defense techniques addressing this issue can be found in the literature (Mozes et al., 2021; Zhou et al., 2019; Wang et al., 2019), results are still limited in performance and text at- tacks keep evolving. This naturally raises concerns around the safe and ethical deployment of NLP systems in real-world processes. Previous research showed that analyzing the model’s logits leads to promising results in dis- criminating manipulated inputs (Wang et al., 2021; Aigrain and Detyniecki, 2019; Hendrycks and Gim- pel, 2016). However, logits-based adversarial de- tectors have been only studied on computer vi- sion applications. Our work transfers this type of methodology to the NLP domain and its contri- bution can be summarized as follows: (1) We introduce a logits-based metric called Word-level Differential Reaction (WDR) captur- ing words with a suspiciously high impact on the classifier. The metric is model-agnostic and also independent from the number of output classes. (2) Based on WDR scores, we train an adversar- ial detector that is able to distinguish original from adversarial input texts preserving syntactical cor- rectness. The approach substantially outperforms the current state of the art in NLP. (3) We show our detector to have full transferabil- ity capabilities and to generalize across multiple datasets, attacks, and target models without need- ing to retrain. Our test configurations include trans- formers and both contextual and genetic attacks. (4) By applying a post-hoc explainability method, we further validate our …

Content truncated to 500 words