Generalized but not Robust? Comparing the Effects of Data Modification Methods on Out-of-Domain Generalization and Adversarial Robustness

Findings of the Association for Computational Linguistics: ACL 2022, pages 2705 - 2718 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Generalized but not Robust? Comparing the Effects of Data Modification Methods on Out-of-Domain Generalization and Adversarial Robustness Tejas Gokhale∗ Swaroop Mishra∗ Man Luo∗ Bhavdeep Singh Sachdeva Chitta Baral Arizona State University {tgokhale, srmishr1, mluo26, bssachde, chitta}@asu.edu Abstract Data modification, either via additional train- ing datasets, data augmentation, debiasing, and dataset filtering, has been proposed as an effective solution for generalizing to out- of-domain (OOD) inputs, in both natural lan- guage processing and computer vision litera- ture. However, the effect of data modification on adversarial robustness remains unclear. In this work, we conduct a comprehensive study of common data modification strategies and evaluate not only their in-domain and OOD performance, but also their adversarial robust- ness (AR). We also present results on a two- dimensional synthetic dataset to visualize the effect of each method on the training distribu- tion. This work serves as an empirical study to- wards understanding the relationship between generalizing to unseen domains and defending against adversarial perturbations. Our findings suggest that more data (either via additional datasets or data augmentation) benefits both OOD accuracy and AR. However, data filter- ing (previously shown to improve OOD accu- racy on natural language inference) hurts OOD accuracy on other tasks such as question an- swering and image classification. We provide insights from our experiments to inform future work in this direction. 1 Introduction Deep neural networks have emerged as a widely popular architectural choice for modeling tasks in multiple domains such as (but not limited to) com- puter vision (Yuille and Liu, 2021), natural lan- guage processing (Hochreiter and Schmidhuber, 1997; Vaswani et al., 2017), and audio (Hannun et al., 2014). While these models are highly capa- ble of learning from training data, recent studies show that they are quite prone to failure on new test sets or under distribution shift (Taori et al., 2020), natural corruptions (Hendrycks and Diet- terich, 2019), adversarial attacks (Goodfellow et al., ∗Equal Contribution 2015), spurious correlations (Beery et al., 2018), and many other types of “unseen” changes that may be encountered after training. This shortcom- ing stems from the i.i.d. assumption in statistical machine learning which guarantees good perfor- mance only on test samples that are drawn from an underlying distribution that is identical to the train- ing dataset. For instance, digit recognition models trained on the black-and-white MNIST training images are almost perfect (> 99% accuracy) on the corresponding test set, yet their performance on colored digits and real-world digits from street number plates is less than 75%. Similarly, state-of- the-art NLP models have been shown to fail when negation is introduced in the input (Kassner and Schütze, 2020). These findings pose a significant challenge to the practical adoption of these models and their reliability in the real-world. To test model performance beyond the tradi- tional notion of in-domain (ID) generalization, two prominent ideas have emerged: out-of-domain (OOD generalization) a.k.a. domain generaliza- …

Content truncated to 500 words