More Identifiable yet Equally Performant Transformers for Text Classification

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 1172–1182 August 1–6, 2021. ©2021 Association for Computational Linguistics 1172 More Identifiable yet Equally Performant Transformers for Text Classification Rishabh Bhardwaj1, Navonil Majumder1, Soujanya Poria1, Eduard Hovy2 1 Singapore University of Technology and Design, Singapore 2 Carnegie Mellon University, Pittsburgh, PA, USA rishabh bhardwaj@mymail.sutd.edu.sg {navonil majumder, sporia}@sutd.edu.sg hovy@cs.cmu.edu Abstract Interpretability is an important aspect of the trustworthiness of a model’s predic- tions. explainedTransformer’s predictions are widely by the attention weights, i.e., a probability distribution generated at its self-attention unit (head). Current empirical studies provide shreds of evidence that attention weights are not explanations by proving that they are not unique. A recent study showed theoretical justifications to this observation by proving the non-identifiability of attention weights. For a given input to a head and its output, if the attention weights generated in it are unique, we call the weights identifiable. In this work, we provide deeper theoretical analysis and empirical observa- tions on the identifiability of attention weights. Ignored in the previous works, we find the at- tention weights are more identifiable than we currently perceive by uncovering the hidden role of the key vector. However, the weights are still prone to be non-unique attentions that make them unfit for interpretation. To tackle this issue, we provide a variant of the encoder layer that decouples the relationship between key and value vector and provides identifiable weights up to the desired length of the input. We prove the applicability of such variations by providing empirical justifications on varied text classification tasks. The implementations are available at https://github.com/declare-lab/ identifiable-transformers. 1 Introduction Widely adopted Transformer architecture (Vaswani et al., 2017) has obviated the need for sequen- tial processing of the input that is enforced in tra- ditional Recurrent Neural Networks (RNN). As a result, compared to a single-layered LSTM or RNN model, a single-layered Transformer model is computationally more efficient, reflecting in a relatively shorter training time (Vaswani et al., 2017). This advantage encourages the training of deep Transformer-based language models on large- scale datasets. Their learning on large corpora has already attained state-of-the-art (SOTA) per- formances in many downstream Natural Language Processing (NLP) tasks. A large number of SOTA machine learning systems even beyond NLP (Lu et al., 2019) are inspired by the building blocks of Transformer that is multi-head self-attention (Rad- ford et al., 2018; Devlin et al., 2018). A model employing an attention-based mechanism generates a probability distribu- tion a = {a1, . . . , an} over the n input units z = {z1, . . . , zn}. The idea is to perform a weighted sum of inputs, denoted by �n i=1 aizi, to produce a more context-involved output. The attention vector, a, are commonly interpreted as scores signifying the relative importance of input units. However, counter-intuitively, it is recently observed that the weights generated in the model do not provide meaningful explanations (Jain and …

Content truncated to 500 words