An Investigation of the (In)effectiveness of Counterfactually Augmented Data

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 3668 - 3681 May 22-27, 2022 c⃝2022 Association for Computational Linguistics An Investigation of the (In)effectiveness of Counterfactually Augmented Data Nitish Joshi1 He He1,2 1Department of Computer Science, New York University 2Center for Data Science, New York University nitish@nyu.edu, hhe@nyu.edu Abstract While pretrained language models achieve ex- cellent performance on natural language un- derstanding benchmarks, they tend to rely on spurious correlations and generalize poorly to out-of-distribution (OOD) data. Recent work has explored using counterfactually- augmented data (CAD)—data generated by minimally perturbing examples to flip the ground-truth label—to identify robust features that are invariant under distribution shift. How- ever, empirical results using CAD during train- ing for OOD generalization have been mixed. To explain this discrepancy, through a toy theo- retical example and empirical analysis on two crowdsourced CAD datasets, we show that: (a) while features perturbed in CAD are indeed ro- bust features, it may prevent the model from learning unperturbed robust features; and (b) CAD may exacerbate existing spurious corre- lations in the data. Our results thus show that the lack of perturbation diversity limits CAD’s effectiveness on OOD generalization, calling for innovative crowdsourcing procedures to elicit diverse perturbation of examples. 1 Introduction Large-scale datasets have enabled tremendous progress in natural language understanding (NLU) (Rajpurkar et al., 2016; Wang et al., 2019) with the rise of pretrained language models (Devlin et al., 2019; Peters et al., 2018). Despite this progress, there have been numerous works showing that mod- els rely on spurious correlations in the datasets, i.e. heuristics that are effective on a specific dataset but do not hold in general (McCoy et al., 2019; Naik et al., 2018; Wang and Culotta, 2020). For example, BERT (Devlin et al., 2019) trained on MNLI (Williams et al., 2018) learns the spurious correlation between world overlap and entailment label. A recent promising direction is to collect counterfactually-augmented data (CAD) by ask- ing humans to minimally edit examples to flip their ground-truth label (Kaushik et al., 2020). Figure 1 shows example edits for Natural Language In- ference (NLI). Given interventions on robust fea- tures that “cause” the label to change, the model is expected to learn to disentangle the spurious and robust features. Despite recent attempt to explain the efficacy of CAD by analyzing the underlying causal structure of the data (Kaushik et al., 2021), empirical results on out-of-distribution (OOD) generalization using CAD are mixed. Specifically, Huang et al. (2020) show that CAD does not improve OOD generaliza- tion for NLI; Khashabi et al. (2020) find that for question answering, CAD is helpful only when it is much cheaper to create than standard examples — but Bowman et al. (2020) report that the cost is actually similar per example. In this work, we take a step towards bridging this gap between what theory suggests and what we observe in practice in regards to CAD. An in- tuitive example to illustrate our key observation is …

Content truncated to 500 words