Attending via both Fine-tuning and Compressing

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2152–2161 August 1–6, 2021. ©2021 Association for Computational Linguistics 2152 Attending via both Fine-tuning and Compressing Jie Zhou1,2, Yuanbin Wu2, Qin Chen2, Xuanjing Huang3, and Liang He1,2 1Shanghai Key Laboratory of Multidimensional Information Processing 2School of Computer Science and Technology, East China Normal University jzhou@ica.stc.sh.cn, {ybwu,qchen,lhe}@cs.ecnu.edu.cn 3School of Computer Science, Fudan University xjhuang@fudan.edu.cn Abstract Though being a primary trend for enhancing interpretability of neural networks, attention mechanism’s reliability and validity are still under debate. In this paper, we try to pu- rify attention scores to obtain a more faithful explanation of downstream models. Specifi- cally, we propose a framework consisting of a learner and a compressor, which performs fine- tuning and compressing iteratively to enhance the performance and interpretability of the at- tention mechanism. The learner focuses on learning better text representations to achieve good decisions by fine-tuning, while the com- pressor aims to perform compressions over the representations to retain the most useful clues for explanations with a Variational in- formation bottleneck ATtention (VAT) mecha- nism. Extensive experiments on eight bench- mark datasets show the great advantages of our proposed approach in terms of both perfor- mance and interpretability. 1 Introduction Attention mechanisms (Bahdanau et al., 2014) have achieved great success in various natural language processing (NLP) tasks. They are introduced to mimic the human eye focusing on important parts in the inputs when predicting labels. The existing studies show attention mechanisms can improve not only the performance but also the interpretabil- ity of the models (Mullenbach et al., 2018; Xie et al., 2017; Xu et al., 2015). Li et al. (2016) pointed the view: “Attention provides an impor- tant way to explain the workings of neural models”. Additionally, Wiegreffe and Pinter (2019) showed that attention mechanisms could help understand the inner workings of a model. The basic assumption of understanding of mod- els with attention scores is that the inputs (e.g., words) with high attentive weights are essential for making decisions. However, as far as we know, it has not been formally verified. Existing research (Jain and Wallace, 2019) also shows that attention is not explicable, and there are a lot of controversy regarding to the result explanations (Wiegreffe and Pinter, 2019; Jain and Wallace, 2019). Moreover, we find that though the attention mechanism can help improve the performance for text classification in our experiments, it may focus on the irrelevant information. For example, in the sentence “A very funny movie.”, the long short-term memory model with standard attention (LSTM-ATT) infers a cor- rect sentiment label while pays more attention to the irrelevant word “movie”, making the result dif- ficult to explain. In general, the attention weights are only op- timized to encode the task-relevant information while are not restricted to imitate human behavior. In order to enhance the interpretability of the at- tention mechanism, recent studies turn to integrate the human provided explanation signals into the attention models. Rei and Søgaard (2018) regular- ized the attention weights …

Content truncated to 500 words