Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagation

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 4083–4093, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 4083 Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagation Po-Sen Huang† Robert Stanforth†§ Johannes Welbl‡§ Chris Dyer† Dani Yogatama† Sven Gowal† Krishnamurthy Dvijotham† Pushmeet Kohli† †DeepMind ‡University College London {posenhuang, stanforth, cdyer, dyogatama, sgowal, dvij, pushmeet}@google.com {j.welbl}@cs.ucl.ac.uk Abstract Neural networks are part of many contempo- rary NLP systems, yet their empirical suc- cesses come at the price of vulnerability to adversarial attacks. Previous work has used adversarial training and data augmentation to partially mitigate such brittleness, but these are unlikely to find worst-case adversaries due to the complexity of the search space arising from discrete text perturbations. In this work, we approach the problem from the opposite di- rection: to formally verify a system’s robust- ness against a predefined class of adversarial attacks. We study text classification under syn- onym replacements or character flip perturba- tions. We propose modeling these input per- turbations as a simplex and then using Interval Bound Propagation – a formal model verifica- tion method. We modify the conventional log- likelihood training objective to train models that can be efficiently verified, which would otherwise come with exponential search com- plexity. The resulting models show only lit- tle difference in terms of nominal accuracy, but have much improved verified accuracy un- der perturbations and come with an efficiently computable formal guarantee on worst case ad- versaries. 1 Introduction Deep models have been shown to be vulnerable against adversarial input perturbations (Szegedy et al., 2013; Kurakin et al., 2016). Small, seman- tically invariant input alterations can lead to dras- tic changes in predictions, leading to poor perfor- mance on adversarially chosen samples. Recent work (Jia and Liang, 2017; Belinkov and Bisk, 2018; Ettinger et al., 2017) also exposed the vul- nerabilities of neural NLP models, e.g. with small ‡Work done during an internship at DeepMind. §Equal contribution. fantastic Conv ReLU FC FC Input perturbations Interval bounds Propagated regions Decision boundary logits conference occasion good great event great event Upper bounds Figure 1: Illustration of verification with the input simplex and Interval Bound Propagation. From the left, input perturbations define the extreme points of a simplex (in red, projected to 2D here) around the statement “great event” that is propagated through a model. At each layer, this shape deforms itself, but can be bounded by axis-parallel bounding boxes, which are propagated similarly. Finally, in logit space, we can compute an upper bound on the worst-case specifica- tion violation (e.g., prediction changes). character perturbations (Ebrahimi et al., 2018) or paraphrases (Ribeiro et al., 2018; Iyyer et al., 2018). These adversarial attacks highlight often unintu- itive model failure modes and present a challenge to deploying NLP models. Common attempts to mitigate the issue are ad- versarial training (Ebrahimi et al., 2018) and data augmentation (Belinkov and Bisk, 2018; Li et al., 2017), which lead …

Content truncated to 500 words