LISA: Explaining Recurrent Neural Network Judgments via Layer-wIse Semantic Accumulation and Example to Pattern Transformation

Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 154–164 Brussels, Belgium, November 1, 2018. c⃝2018 Association for Computational Linguistics 154 LISA: Explaining Recurrent Neural Network Judgments via Layer-wIse Semantic Accumulation and Example to Pattern Transformation Pankaj Gupta1,2, Hinrich Sch¨utze2 1Corporate Technology, Machine-Intelligence (MIC-DE), Siemens AG Munich, Germany 2CIS, University of Munich (LMU) Munich, Germany pankaj.gupta@siemens.com pankaj.gupta@campus.lmu.de | inquiries@cislmu.org Abstract Recurrent neural networks (RNNs) are tem- poral networks and cumulative in nature that have shown promising results in various nat- ural language processing tasks. Despite their success, it still remains a challenge to under- stand their hidden behavior. In this work, we analyze and interpret the cumulative na- ture of RNN via a proposed technique named as Layer-wIse-Semantic-Accumulation (LISA) for explaining decisions and detecting the most likely (i.e., saliency) patterns that the net- work relies on while decision making. We demonstrate (1) LISA: “How an RNN accumu- lates or builds semantics during its sequential processing for a given text example and ex- pected response” (2) Example2pattern: “How the saliency patterns look like for each cate- gory in the data according to the network in de- cision making”. We analyse the sensitiveness of RNNs about different inputs to check the increase or decrease in prediction scores and further extract the saliency patterns learned by the network. We employ two relation classifi- cation datasets: SemEval 10 Task 8 and TAC KBP Slot Filling to explain RNN predictions via the LISA and example2pattern. 1 Introduction The interpretability of systems based on deep neu- ral network is required to be able to explain the reasoning behind the network prediction(s), that offers to (1) verify that the network works as ex- pected and identify the cause of incorrect deci- sion(s) (2) understand the network in order to im- prove data or model with or without human in- tervention. There is a long line of research in techniques of interpretability of Deep Neural net- works (DNNs) via different aspects, such as ex- plaining network decisions, data generation, etc. Erhan et al. (2009); Hinton (2012); Simonyan et al. (2013) and Nguyen et al. (2016) focused on model aspects to interpret neural networks via activa- tion maximization approach by finding inputs that maximize activations of given neurons. Goodfel- low et al. (2014) interprets by generating adversar- ial examples. However, Baehrens et al. (2010) and Bach et al. (2015); Montavon et al. (2017) explain neural network predictions by sensitivity analysis to different input features and decomposition of decision functions, respectively. Recurrent neural networks (RNNs) (Elman, 1990) are temporal networks and cumulative in nature to effectively model sequential data such as text or speech. RNNs and their variants such as LSTM (Hochreiter and Schmidhuber, 1997) have shown success in several natural language processing (NLP) tasks, such as entity extraction (Lample et al., 2016; Ma and Hovy, 2016), rela- tion extraction (Vu et al., 2016a; Miwa and Bansal, 2016; Gupta et al., 2016, 2018c), language mod- eling (Mikolov et al., 2010; Peters et al., 2018), …

Content truncated to 500 words