A Geometry-Inspired Attack for Generating Natural Language Adversarial Examples

Proceedings of the 28th International Conference on Computational Linguistics, pages 6679–6689 Barcelona, Spain (Online), December 8-13, 2020 6679 A Geometry-Inspired Attack for Generating Natural Language Adversarial Examples Zhao Meng, Roger Wattenhofer Department of Electrical Engineering and Information Technology ETH Zurich, Switzerland {zhmeng, wattenhofer}@ethz.ch Abstract Generating adversarial examples for natural language is hard, as natural language consists of discrete symbols, and examples are often of variable lengths. In this paper, we propose a geometry- inspired attack for generating natural language adversarial examples. Our attack generates adversarial examples by iteratively approximating the decision boundary of Deep Neural Networks (DNNs). Experiments on two datasets with two different models show that our attack fools natural language models with high success rates, while only replacing a few words. Human evaluation shows that adversarial examples generated by our attack are hard for humans to recognize. Further experiments show that adversarial training can improve model robustness against our attack. 1 Introduction Although Deep Neural Networks (DNNs) have been successful in many machine learning applica- tions (Kim, 2014; Rajpurkar et al., 2016; He et al., 2016), researchers have demonstrated that DNNs are remarkably vulnerable to adversarial attacks, which generate adversarial examples by adding small perturbations to the original input (Szegedy et al., 2014; Goodfellow et al., 2015; Nguyen et al., 2015). Adversarial examples are essential as they showcase the limitations of DNN models. Like humans, good DNN models should be robust to small perturbations to inputs. If a DNN model judges two almost identical inputs differently, one must profoundly question the quality of the DNN. As such adversarial examples are more than just a gimmick: they are a proof of the fundamental limitations of a DNN model. Previous research on adversarial attacks has been largely focused on images, e.g., (Akhtar and Mian, 2018). In this paper, we study how to adversarially attack natural language models. Generating adversarial examples for natural language is fundamentally different from generating adversarial examples for images. Images live in a continuous universe, where one can simply change pixel values. Natural language sentences and words on the other hand are typically discrete. This discrete nature makes it difficult to apply existing attacks from the image domain directly to natural language, as an arbitrary point in the input space is unlikely to correspond to a valid natural language sentence or word. Moreover, inputs of natural language to DNNs are of variable lengths, which further complicates generating adversarial examples for natural language. Despite these obstacles, researchers have proposed various attacks to generate adversarial examples for natural language. Jia and Liang (2017) manage to fool a DNN model for machine reading by adding sentences to the original texts. Zhao et al. (2018) generate adversarial examples for natural language by using an autoencoder. Ebrahimi et al. (2018) propose a gradient-based attack to generate adversarial examples in the granularity of individual characters. Zhang et al. (2019) generate fluent adversarial examples using Metropolis-Hastings sampling. Ren et al. (2019) combine several heuristics to generate adversarial examples. However, all these methods do not …

Content truncated to 500 words