An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models

An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models Lifu Tu1 ∗ Garima Lalwani2 Spandana Gella2 He He3 ∗ 1Toyota Technological Institute at Chicago 2Amazon AI 3New York University lifu@ttic.edu, {glalwani, sgella}@amazon.com, hehe@cs.nyu.edu Abstract Recent work has shown that pre-trained lan- guage models such as BERT improve robust- ness to spurious correlations in the dataset. Intrigued by these results, we find that the key to their success is generalization from a small amount of counterexamples where the spurious correlations do not hold. When such minority examples are scarce, pre-trained mod- els perform as poorly as models trained from scratch. In the case of extreme minority, we propose to use multi-task learning (MTL) to improve generalization. Our experiments on natural language inference and paraphrase identification show that MTL with the right auxiliary tasks significantly improves perfor- mance on challenging examples without hurt- ing the in-distribution performance. Further, we show that the gain from MTL mainly comes from improved generalization from the minor- ity examples. Our results highlight the impor- tance of data diversity for overcoming spurious correlations.1 1 Introduction A key challenge in building robust NLP models is the gap between limited linguistic variations in the training data and the diversity in real-world languages. Thus models trained on a specific dataset are likely to rely on spurious correlations: prediction rules that work for the majority examples but do not hold in general. For example, in naturallanguage inference (NLI) tasks, previous work has found that models learned on notable benchmarks achieve high accuracy by associating ∗Most work was done during first author’s internship and last author’s work at Amazon AI. 1Code is available at https://github.com/lifu- tu/Study-NLP-Robustness. high word overlap between the premise and the hypothesis with entailment (Dasgupta et al., 2018; McCoy et al., 2019). Consequently, these models perform poorly on the so-called challenging or adversarial datasets, where such correlations no longer hold (Glockner et al., 2018; McCoy et al., 2019; Nie et al., 2019; Zhang et al., 2019). This issue has also been referred to as annotation arti- facts (Gururangan et al., 2018), dataset bias (He et al., 2019; Clark et al., 2019), and group shift (Oren et al., 2019; Sagawa et al., 2020) in the literature. Most current methods rely on prior knowledge of spurious correlations in the dataset and tend to suffer from a trade-off between in-distribution accuracy on the independent and identically distributed (i.i.d.) test set and robust accuracy2 on the challenging dataset. Nevertheless, recent em- pirical results have suggested that self-supervised pre-training improves robust accuracy, while not using any task-specific knowledge nor incurring in-distribution accuracy drop (Hendrycks et al., 2019, 2020). In this paper, we aim to investigate how and when pre-trained language models such as BERT improve performance on challenging datasets. Our key finding is that pre-trained models are more robust to spurious correlations becausethey can gen- eralize from a minority of training examples that counter the spurious pattern, e.g., non-entailment examples with high premise-hypothesis word overlap. Specifically, removing these counterex- amples from …

Content truncated to 500 words