DEMETR: Diagnosing Evaluation Metrics for Translation

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 976 - 991 December 7-11, 2022 ©2022 Association for Computational Linguistics “Will You Find These Shortcuts?” A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classification Jasmijn Bastings Sebastian Ebert∗ Polina Zablotskaia∗ Anders Sandholm Katja Filippova Google Research {bastings,eberts,polinaz,sandholm,katjaf}@google.com Abstract Feature attribution a.k.a. input salience meth- ods which assign an importance score to a feature are abundant but may produce surpris- ingly different results for the same model on the same input. While differences are expected if disparate definitions of importance are as- sumed, most methods claim to provide faith- ful attributions and point at the features most relevant for a model’s prediction. Existing work on faithfulness evaluation is not conclu- sive and does not provide a clear answer as to how different methods are to be compared. Fo- cusing on text classification and the model de- bugging scenario, our main contribution is a protocol for faithfulness evaluation that makes use of partially synthetic data to obtain ground truth for feature importance ranking. Follow- ing the protocol, we do an in-depth analysis of four standard salience method classes on a range of datasets and lexical shortcuts for BERT and LSTM models. We demonstrate that some of the most popular method config- urations provide poor results even for simple shortcuts while a method judged to be too sim- plistic works remarkably well for BERT. 1 Introduction A prominent class of explainability techniques as- sign salience scores to the input features, which reflect the importance of the features to the model’s decision. When applied to text classifiers those methods produce highlights over the input (sub)words. Interestingly, different methods may produce surprisingly dissimilar highlights. Figure 1 shows this using the Language Interpretability Tool (Tenney et al., 2020). So a natural question is: which method should one use? While a method whose highlights happen to look plausible may facilitate a task like text annotation (Pavlopoulos et al., 2017; Strout et al., 2019; Schmidt and Biess- mann, 2019), many salience methods seem to be ∗Equal contribution. Figure 1: Salience maps produced by four common methods on a sentiment classification example (SST2) for a BERT model. The same token (eastwood) is as- signed the highest (Grad-L2), the lowest (GxI, LIME) and a mid-range (IG) importance score (color intensity indicates salience; blue and purple stand for positive, red stands for negative weights). A developer investi- gating a hypothesis about specific named entities being associated with the label would probably be unsure as to whether the example provides support for or against the hypothesis. motivated by the debugging scenario where faith- fulness to the model’s reasoning is a requirement (Jacovi and Goldberg, 2020). Indeed, known suc- cess stories from input salience methods in domains other than language are similar in that they teach us a lesson of not trusting a classifier based on its stellar performance on a standard test set. In the medical domain, for example, heatmaps over im- ages helped …

Content truncated to 500 words