Adversarial Attack and Defense of Structured Prediction Models

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 2327–2338, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 2327 Adversarial Attack and Defense of Structured Prediction Models Wenjuan Han1∗, Liwen Zhang2,3,4,5∗, Yong Jiang6, Kewei Tu2,3† 1School of Computing, National University of Singapore, Singapore 2School of Information Science and Technology, ShanghaiTech University 3Shanghai Engineering Research Center of Intelligent Vision and Imaging 4Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences 5University of Chinese Academy of Sciences 6Alibaba DAMO Academy, Alibaba Group dcshanw@nus.edu.sg {zhanglw, tukw}@shanghaitech.edu.cn yongjiang.jy@alibaba-inc.com Abstract Building an effective adversarial attacker and elaborating on countermeasures for adversar- ial attacks for natural language processing (NLP) have attracted a lot of research in re- cent years. However, most of the existing ap- proaches focus on classification problems. In this paper, we investigate attacks and defenses for structured prediction tasks in NLP. Besides the difficulty of perturbing discrete words and the sentence fluency problem faced by attack- ers in any NLP tasks, there is a specific chal- lenge to attackers of structured prediction mod- els: the structured output of structured pre- diction models is sensitive to small perturba- tions in the input. To address these prob- lems, we propose a novel and unified frame- work that learns to attack a structured pre- diction model using a sequence-to-sequence model with feedbacks from multiple reference models of the same structured prediction task. Based on the proposed attack, we further rein- force the victim model with adversarial train- ing, making its prediction more robust and ac- curate. We evaluate the proposed framework in dependency parsing and part-of-speech tag- ging. Automatic and human evaluations show that our proposed framework succeeds in both attacking state-of-the-art structured prediction models and boosting them with adversarial training. 1 Introduction Adversarial examples, which contain perturbations to the input of a model that elicit large changes in the output, have been shown to be an effective way of assessing the robustness of models in nat- ural language processing (NLP) (Jia and Liang, 2017; Belinkov and Bisk, 2018; Hosseini et al., 2017; Samanta and Mehta, 2017; Alzantot et al., ∗Equal contributions. †Corresponding author. I am a writer x Parser A ˆx Parser A min r log P(y∗|x + r) Gold parse BP min r log P(y =?|ˆx + r) Unknown parse ˆx1 ˆx2 ˆx3 x1 x2 x3 x4 I fire a writer ˆx4 x1 x2 x3 x4 ˆx4 ˆx3 ˆx2 ˆx1 Figure 1: An example showing the challenges in attack- ing a dependency parser using gradient-based methods. A small perturbation to the sentence x changes one word from “am” to “fires”. This change makes the per- turbed example I fires a writer ungrammatical. Even if the perturbed example is “I fire a writer” that meets the rules of grammar, the true output structure is still different from the input sentence “I am a writer”. More importantly, this true parse is unknown to the attacker, which hinders the next update step. 2018; Ebrahimi et al., 2018; Michel et al., 2019; …

Content truncated to 500 words