Informativeness and Invariance: Two Perspectives on Spurious Correlations in Natural Language

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4326 - 4331 July 10-15, 2022 ©2022 Association for Computational Linguistics Informativeness and Invariance: Two Perspectives on Spurious Correlations in Natural Language Jacob Eisenstein Google Research jeisenstein@google.com Abstract Spurious correlations are a threat to the trust- worthiness of natural language processing sys- tems, motivating research into methods for identifying and eliminating them. However, addressing the problem of spurious correla- tions requires more clarity on what they are and how they arise in language data. Gardner et al. (2021) argue that the compositional na- ture of language implies that all correlations between labels and individual “input features” are spurious. This paper analyzes this proposal in the context of a toy example, demonstrat- ing three distinct conditions that can give rise to feature-label correlations in a simple PCFG. Linking the toy example to a structured causal model shows that (1) feature-label correlations can arise even when the label is invariant to in- terventions on the feature, and (2) feature-label correlations may be absent even when the la- bel is sensitive to interventions on the feature. Because input features will be individually cor- related with labels in all but very rare circum- stances, domain knowledge must be applied to identify spurious correlations that pose gen- uine robustness threats. 1 Introduction Spurious correlations are a growing source of con- cern in machine learning (Geirhos et al., 2020) and related fields including natural language process- ing (Gururangan et al., 2018; McCoy et al., 2019, inter alia). While the intuition is fairly clear — spurious correlations are features that are useful in the training data but unreliable in general — the notion is frequently referenced without a formal definition. Gardner et al. (2021) propose a defini- tion in terms of conditional probabilities: a feature Xi is spuriously correlated with the label Y unless P(Y | Xi) is uniform. The definition can be gener- alized from uniformity to independence (Xi ⊥⊥ Y ) without affecting the claims of the paper. They go on to argue that “in a language understanding S ˜Zpizza ˜Y+ �The pizza �� � X1 was not � �� � X2 �too greasy �� � X3 Z Y Figure 1: An instance from the toy model. The up- per part of the figure corresponds to fX, the function that generates the text via a PCFG (see fig. 2): nodes represent non-terminals in the grammar and edges rep- resent context-free derivations. The lower part of the figure corresponds to the causal model of the sentiment Y and target Z. Here nodes represent random variables and edges represent causal relationships. problem, ...all simple correlations between input features and output labels are spurious” (emphasis in the original). The property that individual input features should be independent of labels — which I will call marginally uninformative input features (UIF)1 — is treated as an assumption about the nature of language processing and also as a desider- atum that …

Content truncated to 500 words