Contrastive Explanations for Model Interpretability

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1597–1611 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 1597 Contrastive Explanations for Model Interpretability Alon Jacovi♥∗ Swabha Swayamdipta♣ Shauli Ravfogel♥♣ Yanai Elazar♥♣ Yejin Choi♦♣ Yoav Goldberg♥♣ ♥Bar Ilan University ♣Allen Institute for Artificial Intelligence ♦Paul G. Allen School of Computer Science and Engineering, University of Washington alonjacovi@gmail.com {swabhas,shaulir,yanaie,yoavg,yejinc}@allenai.org Abstract Contrastive explanations clarify why an event occurred in contrast to another. They are inher- ently intuitive to humans to both produce and comprehend. We propose a method to produce contrastive explanations in the latent space, via a projection of the input representation, such that only the features that differentiate two po- tential decisions are captured. Our modifica- tion allows model behavior to consider only contrastive reasoning, and uncover which as- pects of the input are useful for and against particular decisions. Additionally, for a given input feature, our contrastive explanations can answer for which label, and against which al- ternative label, is the feature useful. We pro- duce contrastive explanations via both high- level abstract concept attribution and low-level input token/span attribution for two NLP clas- sification benchmarks. Our findings demon- strate the ability of label-contrastive explana- tions to provide fine-grained interpretability of model decisions.1 1 Introduction Explanations in machine learning attempt to un- cover the causal factors leading to a model’s deci- sions. Methods for producing model explanations often seek all causal factors at once—making them difficult to comprehend—or organize the factors via heuristics, such as gradient saliency (Simonyan et al., 2013; Li et al., 2016). However, it remains unclear what makes a particular collection of causal factors a good explanation. Studies in social science establish that human explanations, conversely, are typically “contrastive” (Miller, 2019): they rely on the causal factors that explain why an event occurred instead of an alter- native event (Lipton, 1990). Such explanations pro- ∗Work done during an internship at the Allen Institute for Artificial Intelligence. 1Code and data are available at https://github. com/allenai/contrastive-explanations. He has 25 years of experience. Dr. Ismaili is affiliated with Medical Center Of Arlington. His specialties include Oral and Maxillofacial Surgery. He speaks English. (1) Why are they a dentist? Dentist (2) Why are they a dentist rather than an accountant? (3) Why are they a dentist rather than a surgeon? He has 25 years of experience. Dr. Ismaili is affiliated with Medical Center Of Arlington. His specialties include Oral and Maxillofacial Surgery. He speaks English. He has 25 years of experience. Dr. Ismaili is affiliated with Medical Center Of Arlington. His specialties include Oral and Maxillofacial Surgery. He speaks English. He has 25 years of experience. Dr. Ismaili is affiliated with Medical Center Of Arlington. His specialties include Oral and Maxillofacial Surgery. He speaks English. Figure 1: Illustrative example of a biography from BIOS (De-Arteaga et al., 2019) with explanations for the occupation label highlighted in yellow. Explana- tions without an explicit contrast (1) are potentially misaligned with human expectations of what is being explained, making …

Content truncated to 500 words