Explaining Classes through Stable Word Attributions

Findings of the Association for Computational Linguistics: ACL 2022, pages 1063 - 1074 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Explaining Classes through Stable Word Attributions Samuel Rönnqvist, Amanda Myntti, Aki-Juhani Kyröläinen Filip Ginter and Veronika Laippala TurkuNLP University of Turku, Finland first.last@utu.fi Abstract Input saliency methods have recently become a popular tool for explaining predictions of deep learning models in NLP. Nevertheless, there has been little work investigating meth- ods for aggregating prediction-level explana- tions to the class level, nor has a framework for evaluating such class explanations been es- tablished. We explore explanations based on XLM-R and the Integrated Gradients input at- tribution method, and propose 1) the Stable At- tribution Class Explanation method (SACX) to extract keyword lists of classes in text classifi- cation tasks, and 2) a framework for the sys- tematic evaluation of the keyword lists. We find that explanations of individual predictions are prone to noise, but that stable explanations can be effectively identified through repeated training and explanation. We evaluate on web register data and show that the class explana- tions are linguistically meaningful and distin- guishing of the classes. 1 Introduction In recent years, various approaches to explaining predictions of deep neural networks have been at- tracting interest in the fields of NLP and computer vision (see, Montavon et al. (2018)). Several tech- niques have been suggested in this vein, including model attention visualization (see, e.g., Vig and Be- linkov (2019)), and input attribution (or saliency) methods (see Bastings and Filippova, 2020; Ding and Koehn, 2021; Simonyan et al., 2014), which focus on explaining individual predictions. How- ever, showing how a model perceives larger units such as entire classes in a text classification task would be crucial for gaining a global understanding of deep classifiers and salient word features. Moreover, text classification models often strug- gle to truly generalize (Laippala et al., 2021; Pe- trenz and Webber, 2011). For instance, McCoy et al. (2020) show in repeated experiments with BERT on a text inference task that, while consis- tent test set performance was achieved, the degree of generalization as measured on a related task var- ied significantly, due to randomized initializations of the decision layer and order of training examples. Similarly, Laippala et al. (2021) demonstrate that resampling of the data had a positive impact on feature stability of linear support vector machines. Thus, various random aspects of the training pro- cess may affect the reliability of modeling results, beyond predictive performance on a test set, espe- cially in deep language models. In this paper, we propose a method for explain- ing classes in a text classification task using deep language models based on input attributions esti- mated with the Integrated Gradients (IG) method (Sundararajan et al., 2017). We focus specifically on IG as it provides a general framework for es- timating feature importance in deep neural net- works and has been shown to provide reliable saliency maps in text classification among other tasks. For a discussion on the merits of …

Content truncated to 500 words