Learning Explanations from Language Data

Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 316–318 Brussels, Belgium, November 1, 2018. c⃝2018 Association for Computational Linguistics 316 Learning Explanations from Language Data David Harbecke* Robert Schwarzenberg* German Research Center for Artificial Intelligence (DFKI) Alt-Moabit 91c, 10559 Berlin, Germany {firstname.lastname}@dfki.de Christoph Alt Abstract PatternAttribution is a recent method, intro- duced in the vision domain, that explains classifications of deep neural networks. We demonstrate that it also generates meaningful interpretations in the language domain. 1 Introduction In the last decade, deep neural classifiers achieved state-of-the-art results in many domains, among others in vision and language. Due to the com- plexity of a deep neural model, however, it is dif- ficult to explain its decisions. Understanding its decision process potentially allows to improve the model and may reveal new knowledge about the input. Recently, Kindermans et al. (2018) claimed that “popular explanation approaches for neural net- works (...) do not provide the correct explana- tion, even for a simple linear model.” They show that in a linear model, the weights serve to can- cel noise in the input data and thus the weights show how to extract the signal but not what the signal is. This is why explanation methods need to move beyond the weights, the authors explain, and they propose the methods “PatternNet” and “PatternAttribution” that learn explanations from data. We test their approach in the language do- main and point to room for improvement in the new framework. 2 Methods Kindermans et al. (2018) assume that the data x passed to a linear model wT x = y is composed of signal (s) and noise (d, from distraction) x = s+d. Furthermore, they also assume that there is a linear relation between signal and target yas = s where as is a so called signal base vector, which is in fact the “pattern” that PatternNet finds for us. As mentioned in the introduction, the authors show that in the model above, w serves to cancel the noise such that wT d = 0, wT s = y. (1) They go on to explain that a good signal estima- tor S(x) = ˆs should comply to the conditions in Eqs. 1 but that these alone form an ill-posed qual- ity criterion since S(x) = u(wT u)−1y already satisfies them for any u for which wT u ̸= 0. To address this issue they introduce another quality criterion over a batch of data x: ρ(S) = 1 − max v corr( y ���� wT x, vT ˆd � �� � (x − S(x))) (2) and point out that Eq. 2 yields maximum values for signal estimators that remove most of the in- formation about y in the noise. We argue that Eq. 2 still is not exhaustive. Con- sider the artificial estimator Sm(x) = mx + (1 − m)s = s + md which arguably is a a bad signal estimator for large m as its estimation contains scaled noise, md. Nevertheless, …

Content truncated to 500 words