Targeted Adversarial Training for Natural Language Understanding

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5385–5393 June 6–11, 2021. ©2021 Association for Computational Linguistics 5385 Targeted Adversarial Training for Natural Language Understanding Lis Pereira1∗, Xiaodong Liu2∗, Hao Cheng2, Hoifung Poon2, Jianfeng Gao2, Ichiro Kobayashi1 1 Ochanomizu University 2 Microsoft Research {kanashiro.pereira,kobayashi.ichiro}@ocha.ac.jp {xiaodl,chehao,hoifung,jfgao}@microsoft.com Abstract We present a simple yet effective Targeted Adversarial Training (TAT) algorithm to im- prove adversarial training for natural language understanding. The key idea is to intro- spect current mistakes and prioritize adversar- ial training steps to where the model errs the most. Experiments show that TAT can sig- nificantly improve accuracy over standard ad- versarial training on GLUE and attain new state-of-the-art zero-shot results on XNLI. Our code will be released at: https://github. com/namisan/mt-dnn. 1 Introduction Adversarial training has proven effective in improv- ing model generalization and robustness in com- puter vision (Madry et al., 2017; Goodfellow et al., 2014) and natural language processing (NLP) (Zhu et al., 2019; Jiang et al., 2019; Cheng et al., 2019; Liu et al., 2020a; Pereira et al., 2020; Cheng et al., 2020). It works by augmenting the input with a small perturbation to steer the current model pre- diction away from the correct label, thus forcing subsequent training to make the model more robust and generalizable. Aside from some prior work in computer vision (Dong et al., 2018; Tramèr et al., 2017), most adversarial training approaches adopt non-targeted attacks, where the model prediction is not driven towards a specific incorrect label. In NLP, the cutting-edge research in adversarial train- ing tends to focus on making adversarial training less expensive (e.g., by reusing backward steps in FreeLB (Zhu et al., 2019)) or regularizing rather than replacing the standard training objective (e.g., in virtual adversarial training (VAT) (Jiang et al., 2019)). By contrast, in this paper, we investigate an or- thogonal direction by augmenting adversarial train- ing with introspection capability and adopting tar- geted attacks to focus on where the model errs the ∗Equal contribution. (a) BERT with standard fine-tuning (b) BERT with TAT fine-tuning Figure 1: Comparison of confusion matrices on MNLI development set (in-domain). X-axis and Y-axis repre- sent the predicted and gold labels, respectively. TAT produces an accuracy gain of 1.7 absolute points. most. We observe that in many NLP applications, the error patterns are non-uniform. For example, in the MNLI development set (in-domain), stan- dard fine-tuned BERT model tends to misclassify a non-neutral instance as “neutral” more often than the opposite label (Figure 1 top). We thus propose Targeted Adversarial Training (TAT), a simple yet effective algorithm for adversarial training. For each instance, instead of taking adversarial steps away from the gold label, TAT samples an incor- rect label proportional to how often the current 5386 model makes the same error in general, and takes adversarial steps towards the chosen incorrect la- bel. To our knowledge, this is the first attempt to apply targeted adversarial training to NLP tasks. In our experiments, this …

Content truncated to 500 words