Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3917–3930 June 6–11, 2021. ©2021 Association for Computational Linguistics 3917 Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks Hanjie Chen1 Song Feng2 Jatin Ganhotra2 Hui Wan2 Chulaka Gunasekara2 Sachindra Joshi2 Yangfeng Ji1 1Department of Computer Science, University of Virginia, Charlottesville, VA, USA 2IBM Research AI {hc9mx, yangfeng}@virginia.edu {sfeng, jatinganhotra, hwan}@us.ibm.com {Chulaka.Gunasekara@, jsachind@in.}ibm.com Abstract Explaining neural network models is impor- tant for increasing their trustworthiness in real- world applications. Most existing methods generate post-hoc explanations for neural net- work models by identifying individual feature attributions or detecting interactions between adjacent features. However, for models with text pairs as inputs (e.g., paraphrase identifi- cation), existing methods are not sufficient to capture feature interactions between two texts and their simple extension of computing all word-pair interactions between two texts is computationally inefficient. In this work, we propose the Group Mask (GMASK) method to implicitly detect word correlations by group- ing correlated words from the input text pair together and measure their contribution to the corresponding NLP tasks as a whole. The pro- posed method is evaluated with two different model architectures (decomposable attention model and BERT) across four datasets, includ- ing natural language inference and paraphrase identification tasks. Experiments show the ef- fectiveness of GMASK in providing faithful explanations to these models 1. 1 Introduction Explaining deep neural networks is critical for re- vealing their prediction behaviors and enhancing the trustworthiness of applying them in real-world applications. Many methods have been proposed to explain neural network models from the post- hoc manner that generates faithful explanations based on model predictions (Ribeiro et al., 2016; Lundberg and Lee, 2017; Sundararajan et al., 2017; Guidotti et al., 2018). Most existing work focuses on identifying word attributions (Rocktäschel et al., 2015; Li et al., 2016; Thorne et al., 2019) for NLP tasks. Knowing which individual features are im- portant might not be enough for explaining model 1Code for this paper is available at https://github. com/UVa-NLP/GMASK Figure 1: An illustration of obtaining individual word attributions (Indiv. Attr.) and weighted word attribu- tions (Weighted Attr.), where the color of each block represents word importance or group importance, and the color saturation of purple lines indicates the proba- bility of a word belonging to a specific group. behaviors. Then, other recent work exploits fea- ture interactions as explanations (Singh et al., 2018; Chen et al., 2020; Tsang et al., 2020). However, they could suffer computation inefficiency while computing interactions between all word pairs, and they also fall short for identifying multiple impor- tant words correlated from different input sources for predictions. Such intuitions are particularly im- portant for explaining sentence pair modeling tasks such as natural language inference (NLI) (Bowman et al., 2015) and paraphrase identification (PI) (Yin and Schütze, 2015). Figure 1 shows an example of NLI, where the model makes correct prediction as CONTRADIC- TION. The first column visualizes individual word attributions to the …

Content truncated to 500 words