Combining Feature and Instance Attribution to Detect Artifacts

Findings of the Association for Computational Linguistics: ACL 2022, pages 1934 - 1946 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Combining Feature and Instance Attribution to Detect Artifacts Pouya Pezeshkpour University of California, Irvine pezeshkp@uci.edu Sarthak Jain Northeastern University jain.sar@northeastern.edu Sameer Singh University of California, Irvine sameer@uci.edu Byron C. Wallace Northeastern University b.wallace@northeastern.edu Abstract Training the deep neural networks that domi- nate NLP requires large datasets. These are of- ten collected automatically or via crowdsourc- ing, and may exhibit systematic biases or an- notation artifacts. By the latter we mean spu- rious correlations between inputs and outputs that do not represent a generally held causal re- lationship between features and classes; mod- els that exploit such correlations may appear to perform a given task well, but fail on out of sample data. In this paper we evaluate use of different attribution methods for aid- ing identification of training data artifacts. We propose new hybrid approaches that combine saliency maps (which highlight “important” in- put features) with instance attribution meth- ods (which retrieve training samples “influen- tial” to a given prediction). We show that this proposed training-feature attribution can be used to efficiently uncover artifacts in train- ing data when a challenging validation set is available. We also carry out a small user study to evaluate whether these methods are useful to NLP researchers in practice, with promis- ing results. We make code for all methods and experiments in this paper available.1 1 Introduction Deep networks dominate NLP applications and are being increasingly deployed in the real-world. But what exactly are such models “learning”? One concern is that they may be exploiting artifacts or spurious correlations between inputs and outputs that are present in the training data, but not reflec- tive of the underlying task that the data is intended to represent. We assess the utility of attribution methods for purposes of aiding practitioners in identifying train- Warning: This paper contains examples with texts that might be considered offensive. 1https://github.com/pouyapez/artifact_detection training data … Model Train “yo! that’s sick” Test instance Predict Toxic “yo! that’s sick” Input saliency Instance attribution “shut up!” Toxic Training-feature attribution Toxic Attribution methods Artifact discovery “shut up!” Figure 1: Use of different attribution techniques for ar- tifact discovery in train data. Here attribution methods can reveal inappropriate reliance on certain tokens (e.g., “!”, “yo”) to predict Tweet toxicity; these are artifacts. ing data artifacts, drawing inspiration from prior ef- forts that have suggested the use of attribution meth- ods for this purpose (Han et al., 2020; Zhou et al., 2021). Attribution methods are model-centric; our evaluation of them for artifact discovery there- fore complements recent work on data-centric approaches (Gardner et al., 2021). We consider two families of attribution methods: (1) feature- attribution, which highlight constituent input fea- tures (e.g., tokens) in proportion to their “impor- tance” for an output (Ribeiro et al., 2016; Lund- berg and Lee, 2017; Adebayo et al., 2018), and; (2) instance attribution, which retrieves training instances most responsible for a given …

Content truncated to 500 words