Interpretable Textual Neuron Representations for NLP

Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 325–327 Brussels, Belgium, November 1, 2018. c⃝2018 Association for Computational Linguistics 325 Interpretable Textual Neuron Representations for NLP Nina Poerner, Benjamin Roth, Hinrich Sch¨utze Center for Information and Language Processing LMU Munich, Germany poerner@cis.lmu.de Abstract Input optimization methods, such as Google Deep Dream, create interpretable representa- tions of neurons for computer vision DNNs. We propose and evaluate ways of transfer- ring this technology to NLP. Our results sug- gest that gradient ascent with a gumbel soft- max layer produces n-gram representations that outperform naive corpus search in terms of target neuron activation. The representa- tions highlight differences in syntax awareness between the language and visual models of the Imaginet architecture. 1 Introduction Deep Neural Networks (DNNs) have led to ad- vances in Natural Language Processing, but they are hard to interpret. This is partly due to the fact that their smallest components, i.e., neurons, lack interpretable representations. For computer vision problems, Simonyan et al. (2014) propose to use gradient ascent to find an in- put image that maximizes the activation of a neu- ron of interest. Using these image representations, one can for instance show that lower level neu- rons in vision CNNs specialize in patterns such as stripes (Mordvintsev et al., 2015). Applying gradient ascent input optimization to NLP is not straightforward, as discrete symbols are not open to continuous manipulation. common alternative approach is to search exist-A ing corpora for optimal documents or n-grams (e.g., K´ad´ar et al. (2017), Aubakirova and Bansal (2016)). As this strategy only covers the space of existing inputs, we assume that it may lead to in- correct assumptions. For instance, the represen- tation of a given neuron may suggest that syntax was learned, when in reality this is due to a lack of ungrammatical inputs in the corpus. In the following, we propose and test methods for gradient ascent input optimization in NLP. Our quantitative assessment suggests that one method, which is based on the gumbel softmax trick, pro- duces inputs that are more highly activating than corpus search. By applying this method to the Imaginet architecture, we confirm that a language model pays attention to syntax to some degree, while a visual model looks for key content words and ignores function words. 2 Input optimization for NLP In the following, we denote as f(E) the activation of some neuron of interest when forward-feeding a sequence of embedding vectors E = [e1 . . . eT ]. 2.1 Embedding optimization One straightforward approach to NLP input opti- mization is to treat E like Simonyan et al. (2014) treat images, i.e., to apply gradient ascent directly to the embedding vectors, while keeping other model parameters constant: argmaxE � f(E) However, there is no guarantee that the optimal.� vectors will correspond to the embedding vectors of real words, or even be close to them. In our experiments, the average cosine proximity to the closest real-word embedding is 0.24, suggesting that …

Content truncated to 500 words