Saliency Learning: Teaching the Model Where to Pay Attention

Proceedings of NAACL-HLT 2019, pages 4016–4025 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 4016 Saliency Learning: Teaching the Model Where to Pay Attention Reza Ghaeini, Xiaoli Z. Fern, Hamed Shahbazi, Prasad Tadepalli School of Electrical Engineering and Computer Science, Oregon State University 1148 Kelley Engineering Center, Corvallis, OR 97331-5501, USA {ghaeinim, xfern, shahbazh, tadepall}@eecs.oregonstate.edu Abstract Deep learning has emerged as a compelling solution to many NLP tasks with remarkable performances. However, due to their opac- ity, such models are hard to interpret and trust. Recent work on explaining deep models has introduced approaches to provide insights to- ward the model’s behaviour and predictions, which are helpful for assessing the reliabil- ity of the model’s predictions. However, such methods do not improve the model’s reliabil- ity. In this paper, we aim to teach the model to make the right prediction for the right reason by providing explanation training and ensur- ing the alignment of the model’s explanation with the ground truth explanation. Our exper- imental results on multiple tasks and datasets demonstrate the effectiveness of the proposed method, which produces more reliable predic- tions while delivering better results compared to traditionally trained models. 1 Introduction It is unfortunate that our data is often plagued by meaningless or even harmful statistical biases. When we train a model on such data, it is possible that the classifier focuses on irrelevant biases to achieve high performance on the biased data. Re- cent studies demonstrate that deep learning mod- els noticeably suffer from this issue (Agrawal et al., 2016; Wadhwa et al., 2018; Gururangan et al., 2018). Due to the black-box nature of deep models and the high dimensionality of their inher- ent representations, it is difficult to interpret and trust their behaviour and predictions. Recent work on explanation and interpretation has introduced a few approaches (Simonyan et al., 2013; Ribeiro et al., 2016; Lei et al., 2016; Li et al., 2016, 2017; Ghaeini et al., 2018b; Ribeiro et al., 2018) for ex- planation. Such methods provide insights toward the model’s behaviour, which is helpful for detect- ing biases in our models. However, they do not correct them. Here, we investigate how to incor- porate explanations into the learning process to en- sure that our model not only makes correct predic- tions but also makes them for the right reason. Specifically, we propose to train a deep model using both ground truth labels and additional an- notations suggesting the desired explanation. The learning is achieved via a novel method called saliency learning, which regulates the model’s be- haviour using saliency to ensure that the most crit- ical factors impacting the model’s prediction are aligned with the desired explanation. Our work is closely related to Ross et al. (2017), which also uses the gradient/saliency information to regularize model’s behaviour. However, we dif- fer in the following points: 1) Ross et al. (2017) is limited to regularizing model with gradient of the model’s input. In contrast, we extend this concept to …

Content truncated to 500 words