Understanding Attention for Text Classification

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3418–3428 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 3418 Understanding Attention for Text Classification Xiaobing Sun and Wei Lu StatNLP Research Group Singapore University of Technology and Design xiaobing sun@mymail.sutd.edu.sg, luwei@sutd.edu.sg Abstract Attention has been proven successful in many natural language processing (NLP) tasks. Re- cently, many researchers started to investigate the interpretability of attention on NLP tasks. Many existing approaches focused on examin- ing whether the local attention weights could reflect the importance of input representations. In this work, we present a study on under- standing the internal mechanism of attention by looking into the gradient update process, checking its behavior when approaching a lo- cal minimum during training. We propose to analyze for each word token the following two quantities: its polarity score and its attention score, where the latter is a global assessment on the token’s significance. We discuss con- ditions under which the attention mechanism may become more (or less) interpretable, and show how the interplay between the two quan- tities may impact the model performance.1 1 Introduction Attention mechanism (Bahdanau et al., 2015) has been used as an important component across a wide range of NLP models. Typically, an attention layer produces a distribution over input representations to be attended to. Such a distribution is then used for constructing a weighted combination of the inputs, which will then be employed by certain downstream modules. Recently, several research efforts on investigat- ing the interpretability of attention on tasks such as text classification, question answering, and nat- ural language inference (Jain and Wallace, 2019; Wiegreffe and Pinter, 2019; Arras et al., 2019) have been conducted. One of their important arguments was whether the attention distribution could ade- quately reflect the significance of inputs. To answer this question, they designed a series of metrics and 1Supplementary material and code at https:// github.com/richardsun-voyager/UAFTC conducted corresponding experiments. In their ap- proaches, they were mainly observing how the at- tention may impact the outputs on the pre-trained models by changing some elements in the inputs. While such approaches have resulted in interesting findings, the attention mechanism itself remains a black box to us – it is still largely unclear what are the underlying factors that may have an impact on the attention mechanism. When analyzing the results of a typical model with attention on the text classification tasks, we noticed that in some instances, many of the word tokens with large attention weights were adjectives or adverbs which conveyed explicit signals on the underlying class label. On the other hand, in some other instances, we also noticed that such useful words may not always be able to receive significant attention weights, especially under certain config- urations of hyperparameters, making the attention mechanism less interpretable. Such observations lead to several important ques- tions. First, the attention weight for a word token appears to be the relative measurement to its sig- nificance, and is largely local and …

Content truncated to 500 words