Text Processing Like Humans Do: Visually Attacking and Shielding NLP Systems

Proceedings of NAACL-HLT 2019, pages 1634–1647 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 1634 Text Processing Like Humans Do: Visually Attacking and Shielding NLP Systems Steffen Eger†‡, Gözde Gül Şahin†‡, Andreas Rücklé†, Ji-Ung Lee†‡, Claudia Schulz†‡, Mohsen Mesgar†‡, Krishnkant Swarnkar†, Edwin Simpson†, Iryna Gurevych†‡ †Ubiquitous Knowledge Processing Lab (UKP-TUDA) ‡Research Training Group AIPHES Department of Computer Science, Technische Universität Darmstadt †www.ukp.tu-darmstadt.de ‡www.aiphes.tu-darmstadt.de Abstract Visual modifications to text are often used to obfuscate offensive comments in social media (e.g., “!d10t”) or as a writing style (“1337” in “leet speak”), among other scenar- ios. We consider this as a new type of ad- versarial attack in NLP, a setting to which hu- mans are very robust, as our experiments with both simple and more difficult visual perturba- tions demonstrate. We investigate the impact of visual adversarial attacks on current NLP systems on character-, word-, and sentence- level tasks, showing that both neural and non- neural models are, in contrast to humans, ex- tremely sensitive to such attacks, suffering per- formance decreases of up to 82%. We then explore three shielding methods—visual char- acter embeddings, adversarial training, and rule-based recovery—which substantially im- prove the robustness of the models. How- ever, the shielding methods still fall behind performances achieved in non-attack scenarios, which demonstrates the difficulty of dealing with visual attacks. 1 Introduction For humans, visual similarity can play a decisive role for assessing the meaning of characters. Some evidence for these are: the frequent swapping of similar looking characters in Internet slang or abu- sive comments, creative trademark logos, and at- tack scenarios such as domain name spoofing (see examples in Table 1). Recently, some NLP systems have exploited vi- sual features to capture visual relationships among characters in compositional writing systems such as Chinese or Korean (Liu et al., 2017). How- ever, in more general cases, current neural NLP systems have no built-in notion of visual charac- ter similarity. Rather, they either treat characters as discrete units forming a word or they represent characters by randomly initialized embeddings and Internet slang writing style n00b, w!k!p3d!4, 1337 Toxic comments pi§§, ּמiggers, ḟucking Trademark logos/artwork Domain name spoofing http://wíkipedia.org Table 1: Examples of text in which characters have been changed to visually similar ones. update them during training—typically in order to generate a character-based word representation that is robust to morphological variation or spelling mistakes (Ma and Hovy, 2016). Intriguingly, this marked distinction between human and machine processing can be exploited as a blind spot of NLP systems. For example, spammers might send mali- cious emails or post toxic comments to online dis- cussion forums (Hosseini et al., 2017) by visually ‘perturbing’ the input text in such a way that it is still easily recoverable by humans. The issue of exposing and addressing the weak- nesses of deep learning models to adversarial in- puts, i.e., perturbed versions of original input sam- ples, has recently received considerable attention. For instance, Goodfellow et al. (2015) showed that small perturbations in …

Content truncated to 500 words