Explain by Evidence: An Explainable Memory-based Neural Network for Question Answering

Proceedings of the 28th International Conference on Computational Linguistics, pages 5205–5210 Barcelona, Spain (Online), December 8-13, 2020 5205 Explain by Evidence: An Explainable Memory-based Neural Network for Question Answering Quan Tran1, Nhan Dam2, Tuan Lai3, Franck Dernoncourt1 Trung Le2, Nham Le4 and Dinh Phung2 1Adobe Research 2Monash University 3 University of Illinois at Urbana-Champaign 4 University of Waterloo {qtran, franck.dernoncourt}@adobe.com {nhan.dam, trunglm, dinh.phung}@monash.edu tuanml2@illinois.edu, nhamlevan@gmail.com Abstract Interpretability and explainability of deep neural networks are challenging due to their scale, complexity, and the agreeable notions on which the explaining process rests. Previous work, in particular, has focused on representing internal components of neural networks through human- friendly visuals and concepts. On the other hand, in real life, when making a decision, human tends to rely on similar situations and/or associations in the past. Hence arguably, a promising approach to make the model transparent is to design it in a way such that the model explic- itly connects the current sample with the seen ones, and bases its decision on these samples. Grounded on that principle, we propose in this paper an explainable, evidence-based memory network architecture, which learns to summarize the dataset and extract supporting evidences to make its decision. Our model achieves state-of-the-art performance on two popular question answering datasets (i.e. TrecQA and WikiQA). Via further analysis, we show that this model can reliably trace the errors it has made in the validation step to the training instances that might have caused these errors. We believe that this error-tracing capability provides significant benefit in improving dataset quality in many applications. 1 Introduction Interpretability of neural networks is an active research field in machine learning. Deep neural networks might have tens if not hundreds of millions of parameters (Devlin et al., 2019; Liu et al., 2019a) organized into intricate architectures. The sheer amount of parameters and the complexity of the architectures largely prevent human to directly make sense of which concepts and how the network truly learns. The comparative lack of explainable intuition behind deep neural networks might hamper the development and adoption of those models. In certain scenarios, prediction accuracy alone is not sufficient (Caruana et al., 2015; Lapuschkin et al., 2019). For example, as discussed in (Zhang et al., 2018b; Zhang et al., 2018a), it is difficult to trust a deep model even if it has high test set performance given the inherent biases in the dataset. Thus, we argue that interpretability is perhaps one of the keys to accelerate both the development and adoption of deep neural networks. There have been many successful attempts from the research community to make sense of deep mod- els’ prediction. These attempts can be broadly categorized into several classes. One of the major classes concerns with the network visualization techniques, for example, visual saliency representations in con- volutional models (Simonyan et al., 2013; Sundararajan et al., 2017). For recurrent neural networks (RNN), Karpathy et al. (2015) focused on analyzing and visualizing the RNN to explain its ability to keep track of long-range …

Content truncated to 500 words