Aligning Faithful Interpretations with their Social Attribution

Aligning Faithful Interpretations with their Social Attribution Alon Jacovi Bar Ilan University alonjacovi@gmail.com Yoav Goldberg Bar Ilan University and Allen Institute for AI yoav.goldberg@gmail.com Abstract We find that the requirement of model inter- pretations to be faithful is vague and incom- plete. With interpretation by textual highlights as a case study, we present several failure cases. Borrowing concepts from social science, we identify that the problem is a misalignment between the causal chain of decisions (causal attribution) and the attribution of human be- havior to the interpretation (social attribution). We reformulate faithfulness as an accurate attribution of causality to the model, and in- troduce the concept of aligned faithfulness: faithful causal chains that are aligned with their expected social behavior. The two steps of causal attribution and social attribution together complete the process of explaining behavior. With this formalization, we charac- terize various failures of misaligned faithful highlight interpretations, and propose an al- ternative causal chain to remedy the issues. Finally, we implement highlight explanations of the proposed causal format using contras- tive explanations. 1 Introduction When formalizing the desired properties of a quality interpretation of a model or a decision, the NLP community has settled on the key property of faithfulness (Lipton, 2018; Herman, 2017; Wiegreffe and Pinter, 2019; Jacovi and Goldberg, 2020), or how ‘‘accurately’’ the interpretation represents the true reasoning process of the model. A common pattern of achieving faithfulness in interpretation of neural models is via decompo- sition of a model into steps and inspecting the intermediate steps (Doshi-Velez and Kim, 2017, cognitive chunks). For example, neural modular networks (NMNs; Andreas et al., 2016) first build an execution graph out of neural building blocks, and then apply this graph to data. The graph struc- ture is taken to be a faithful interpretation of the model’s behavior, as it describes the computa- tion precisely. Similarly, highlight methods (also called extractive rationales1), decompose a textual prediction problem into first selecting highlighted texts, and then predicting based on the selected words (select-predict, described in Section 2). The output of the selection component is taken to be a faithful interpretation, as we know exactly which words were selected. Similarly, we know that words that were not selected do not participate in the final prediction. However, Subramanian et al. (2020) call NMN graphs not faithful in cases where there is a discrepancy between a building block’s behavior and its name (i.e., expected behavior). Can we better characterize the requirement of faithful interpretations and amend this discrepancy? We take an extensive and critical look at the formalization of faithfulness and of explanations, with textual highlights as an example use-case. In particular, the select-predict formulation for faithful highlights raises more questions than it provides answers:We describe a variety of curious failure cases of such models in Section 4, as well as experimentally validate that the failure cases are indeed possible and do occur in practice. Concretely, the behavior of the selector and predictor in these models do not necessarily line up with expectations …

Content truncated to 500 words