How do humans perceive adversarial text? A reality check on the validity and naturalness of word-based adversarial attacks

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 8822–8836 July 9-14, 2023 ©2023 Association for Computational Linguistics How do humans perceive adversarial text? A reality check on the validity and naturalness of word-based adversarial attacks Salijona Dyrmishi University of Luxembourg salijona.dyrmishi@uni.lu Salah Ghamizi University of Luxembourg salah.ghamizi@uni.lu Maxime Cordy University of Luxembourg maxime.cordy@uni.lu Abstract Natural Language Processing (NLP) models based on Machine Learning (ML) are suscep- tible to adversarial attacks – malicious algo- rithms that imperceptibly modify input text to force models into making incorrect predictions. However, evaluations of these attacks ignore the property of imperceptibility or study it un- der limited settings. This entails that adver- sarial perturbations would not pass any human quality gate and do not represent real threats to human-checked NLP systems. To bypass this limitation and enable proper assessment (and later, improvement) of NLP model robust- ness, we have surveyed 378 human participants about the perceptibility of text adversarial ex- amples produced by state-of-the-art methods. Our results underline that existing text attacks are impractical in real-world scenarios where humans are involved. This contrasts with pre- vious smaller-scale human studies, which re- ported overly optimistic conclusions regarding attack success. Through our work, we hope to position human perceptibility as a first-class success criterion for text attacks, and provide guidance for research to build effective attack algorithms and, in turn, design appropriate de- fence mechanisms. 1 Introduction Like many other machine learning models, Natural Language Processing (NLP) models are susceptible to adversarial attacks. In NLP, these attacks aim to cause failures (e.g. incorrect decisions) in the model by slightly perturbing the input text in such a way that its original meaning is preserved. Research has reported on the potential of adver- sarial attacks to affect real-world models interact- ing with human users, such as Google’s Perspec- tive and Facebook’s fastText (Li et al., 2019)) More generally, these attacks cover various learning tasks including classification and seq2seq (fake news (Li et al., 2020), toxic content (Li et al., 2019), spam messages (Kuchipudi et al., 2020)), style transfer (Qi et al., 2021) and machine translation (Michel et al., 2019)). It is critical to properly assess model robustness against adversarial attacks to design relevant de- fence mechanisms. This is why research has inves- tigated different attack algorithms based on para- phrasing (Iyyer et al., 2018), character-level (Gao et al., 2018; Pruthi et al., 2019) and word-level (Garg and Ramakrishnan, 2020; Ren et al., 2019) perturbations, and made these algorithms available in standardized libraries (Morris et al., 2020b; Zeng et al., 2021). For the many NLP systems that interact with hu- mans, we argue that effective adversarial attacks should produce text that is both valid and natural. Validity refers to the property that humans perceive the same semantic properties of interest1 for an ad- versarial text as for the original text it was produced from. Naturalness refers to the perception that an adversarial text was produced by humans. Adver- sarial texts that …

Content truncated to 500 words