Efficient Shapley Values Estimation by Amortization for Text Classification

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 8666–8680 July 9-14, 2023 ©2023 Association for Computational Linguistics Efficient Shapley Values Estimation by Amortization for Text Classification Chenghao Yang1,4,∗, Fan Yin2, He He3,4, Kai-Wei Chang2,5, Xiaofei Ma4, Bing Xiang4 1 Unversity of Chicago 2 University of California, Los Angeles 3 New York University 4 AWS AI Labs 5 Amazon Alexa AI yangalan1996@gmail.com, fanyin20@cs.ucla.edu {hehea, kaiweic, xiaofeim, bxiang}@amazon.com Abstract Despite the popularity of Shapley Values in ex- plaining neural text classification models, com- puting them is prohibitive for large pretrained models due to a large number of model evalua- tions. In practice, Shapley Values are often esti- mated with a small number of stochastic model evaluations. However, we show that the esti- mated Shapley Values are sensitive to random seed choices – the top-ranked features often have little overlap across different seeds, espe- cially on examples with longer input texts. This can only be mitigated by aggregating thousands of model evaluations, which on the other hand, induces substantial computational overheads. To mitigate the trade-off between stability and efficiency, we develop an amortized model that directly predicts each input feature’s Shapley Value without additional model evaluations. It is trained on a set of examples whose Shapley Values are estimated from a large number of model evaluations to ensure stability. Experi- mental results on two text classification datasets demonstrate that our amortized model esti- mates Shapley Values accurately with up to 60 times speedup compared to traditional methods. Furthermore, the estimated values are stable as the inference is deterministic. We release our code at https://github.com/yangalan123/ Amortized-Interpretability. 1 Introduction Many powerful natural language processing (NLP) models used in commercial systems only allow users to access model outputs. When these systems are applied in high-stakes domains, such as health- care, finance, and law, it is essential to interpret how these models come to their decisions. To this end, post-hoc black-box explanation methods have been proposed to identify the input features that are most critical to model predictions (Ribeiro et al., 2016; Lundberg and Lee, 2017). A famous class of post- ∗Work done during full-time work at AWS AI Seed=1 Seed=2 Figure 1: Heatmaps of explanation scores of an example from Yelp-Polarity based on two runs of KernelSHAP (KS) using different random seeds. KS is run on a fine-tuned BERT model using 200 samples per instance (approx. 3.47s per instance on average using a single A100 GPU, more than 150 times slower than one for- ward inference of the BERT model). The darker each token is, the higher its explanation score. Clearly, inter- pretation results are significantly different when using different seeds. hoc black-box local explanation methods takes ad- vantage of the Shapley Values (Shapley, 1953) to identify important input features, such as Shapley Value Sampling (SVS) (Strumbelj and Kononenko, 2010) and KernelSHAP (KS) (Lundberg and Lee, 2017). These methods typically start by sampling permutations of the input features (“perturbation samples”) and aggregating model output changes over the …

Content truncated to 500 words