DoLFIn: Distributions over Latent Features for Interpretability

Proceedings of the 28th International Conference on Computational Linguistics, pages 1468–1474 Barcelona, Spain (Online), December 8-13, 2020 1468 DoLFIn: Distributions over Latent Features for Interpretability Phong Le∗ Independent Researcher lephong.xyz@gmail.com Willem Zuidema University of Amsterdam zuidema@uva.nl Abstract Interpreting the inner workings of neural models is a key step in ensuring the robustness and trustworthiness of the models, but work on neural network interpretability typically faces a trade- off: either the models are too constrained to be very useful, or the solutions found by the models are too complex to interpret. We propose a novel strategy for achieving interpretability that – in our experiments – avoids this trade-off. Our approach builds on the success of using probability as the central quantity, such as for instance within the attention mechanism. In our architecture, DoLFIn (Distributions over Latent Features for Interpretability), we do no determine beforehand what each feature represents, and features go altogether into an unordered set. Each feature has an associated probability ranging from 0 to 1, weighing its importance for further processing. We show that, unlike attention and saliency map approaches, this set-up makes it straight-forward to compute the probability with which an input component supports the decision the neural model makes. To demonstrate the usefulness of the approach, we apply DoLFIn to text classification, and show that DoLFIn not only provides interpretable solutions, but even slightly outperforms the classical CNN and BiLSTM text classifiers on the SST2 and AG-news datasets. 1 Introduction Having insights into how a trained neural network solves a given task is important, in order to build more robust, trustworthy, and accurate models (Zhou et al., 2016; Gilpin et al., 2018; Poerner et al., 2018; Belinkov et al., 2017). However gaining insights is often challenging because of the large number of parameters and the nonlinear dependencies between components of the model. One approach to ‘opening the blackbox’ is to use saliency maps (Jacovi et al., 2018; Gupta and Sch¨utze, 2018), to examine which input components (e.g., words) are taken into account. Nevertheless salience alone does not tell us how exactly salient components contribute to the decision of a model. For instance, a sentiment-analysis model might mark both “boring” and “wonderful” in “This film would be wonderful, if the beginning wasn’t so boring” as salient, but saliency scores do not reveal the crucial interaction between all the words in the sentence that ultimately let it express a negative sentiment. Alternatively, one can analyse the flow of information through the network, by tracking (relevance) gradients backwards (Arras et al., 2017) or decomposing (forward) contributions to all intermediate quantities (Murdoch et al., 2018; Jumelet et al., 2019). These methods address some of the problems of saliency maps, but produce results that themselves require further interpretation. For instance, such a method might quantify the relative contribution of the 3rd word when processing the 7th word in the 2nd layer of a multilayer LSTM. That quantity, however, does not easily translate to an explanation for the final prediction that the model …

Content truncated to 500 words