Combating Adversarial Misspellings with Robust Word Recognition

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5582–5591 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5582 Combating Adversarial Misspellings with Robust Word Recognition Danish Pruthi Bhuwan Dhingra Carnegie Mellon UniversityZachary C. Lipton Pittsburgh, USA {ddanish, bdhingra}@cs.cmu.edu, zlipton@cmu.edu Abstract To combat adversarial spelling mistakes, we propose placing a word recognition model in front of the downstream classifier. Our word recognition models build upon the RNN semi- character architecture, introducing several new backoff strategies for handling rare and un- seen words. Trained to recognize words cor- rupted by random adds, drops, swaps, and keyboard mistakes, our method achieves 32% relative (and 3.3% absolute) error reduction over the vanilla semi-character model. No- tably, our pipeline confers robustness on the downstream classifier, outperforming both ad- versarial training and off-the-shelf spell check- ers. Against a BERT model fine-tuned for sen- timent analysis, a single adversarially-chosen character attack lowers accuracy from 90.3% to 45.8%. Our defense restores accuracy to 75%1. Surprisingly, better word recognition does not always entail greater robustness. Our analysis reveals that robustness also depends upon a quantity that we denote the sensitivity. 1 Introduction Despite the rapid progress of deep learning tech- niques on diverse supervised learning tasks, these models remain brittle to subtle shifts in the data distribution. Even when the permissible changes are confined to barely-perceptible perturbations, training robust models remains an open challenge. Following the discovery that imperceptible attacks could cause image recognition models to misclas- sify examples (Szegedy et al., 2013), a veritable sub-field has emerged in which authors iteratively propose attacks and countermeasures. For all the interest in adversarial computer vi- sion, these attacks are rarely encountered out- side of academic research. However, adversarial 1All code for our defenses, attacks, and baselines is available at https://github.com/danishpruthi/ Adversarial-Misspellings Alteration Movie Review Label Original A triumph, relentless and beautiful in its downbeat darkness + Swap A triumph, relentless and beuatiful in its downbeat darkness – Drop A triumph, relentless and beautiful in its dwnbeat darkness – + Defense A triumph, relentless and beautiful in its downbeat darkness + + Defense A triumph, relentless and beautiful in its downbeat darkness + Table 1: Adversarial spelling mistakes inducing senti- ment misclassification and word-recognition defenses. misspellings constitute a longstanding real-world problem. Spammers continually bombard email servers, subtly misspelling words in efforts to evade spam detection while preserving the emails’ intended meaning (Lee and Ng, 2005; Fumera et al., 2006). As another example, programmatic censorship on the Internet has spurred communi- ties to adopt similar methods to communicate sur- reptitiously (Bitso et al., 2013). In this paper, we focus on adversarially-chosen spelling mistakes in the context of text classifica- tion, addressing the following attack types: drop- ping, adding, and swapping internal characters within words. These perturbations are inspired by psycholinguistic studies (Rawlinson, 1976; Matt Davis, 2003) which demonstrated that humans can comprehend text altered by jumbling internal char- acters, provided that the first and last characters of each word remain unperturbed. First, …

Content truncated to 500 words