Measuring the Mixing of Contextual Information in the Transformer

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8698 - 8714 December 7-11, 2022 ©2022 Association for Computational Linguistics Measuring the Mixing of Contextual Information in the Transformer Javier Ferrando, Gerard I. Gállego and Marta R. Costa-jussà TALP Research Center, Universitat Politècnica de Catalunya {javier.ferrando.monsonis,gerard.ion.gallego,marta.ruiz}@upc.edu Abstract The Transformer architecture aggregates input information through the self-attention mecha- nism, but there is no clear understanding of how this information is mixed across the en- tire model. Additionally, recent works have demonstrated that attention weights alone are not enough to describe the flow of informa- tion. In this paper, we consider the whole at- tention block –multi-head attention, residual connection, and layer normalization– and de- fine a metric to measure token-to-token inter- actions within each layer. Then, we aggregate layer-wise interpretations to provide input at- tribution scores for model predictions. Experi- mentally, we show that our method, ALTI (Ag- gregation of Layer-wise Token-to-token Inter- actions), provides more faithful explanations and increased robustness than gradient-based methods. 1 Introduction The Transformer (Vaswani et al., 2017) has be- come ubiquitous in different tasks across multiple domains, becoming the architecture of choice for many NLP (Devlin et al., 2019; Brown et al., 2020) and computer vision (Dosovitskiy et al., 2021) tasks. The self-attention mechanism inside the Transformer is in charge of combining contextual information in its intermediate token representa- tions. Attention weights offer a straightforward layer-wise interpretation, as they provide a distri- bution over input units, which is often presented as giving the relative importance of each input. A prominent line of research has investigated the faithfulness of attention weights (Jain and Wallace, 2019; Serrano and Smith, 2019; Pruthi et al., 2020; Wiegreffe and Pinter, 2019; Madsen et al., 2021b) with contradictory conclusions. Some works have studied layer-wise attention patterns by analyzing standard attention (Kovaleva et al., 2019; Clark et al., 2019; Vig and Belinkov, 2019) and effective Gradℓ2 went here just before a movie . the service was fast but that ’ s it . i ordered the mango and shrimp quesadilla . my friend ordered nachos . the food was not good . i and my friend could not finish our food and we had stomach aches immediately . IGℓ2 went here just before a movie . the service was fast but that ’ s it . i ordered the mango and shrimp quesadilla . my friend ordered nachos . the food was not good . i and my friend could not finish our food and we had stomach aches immediately . ALTI went here just before a movie . the service was fast but that ’ s it . i ordered the mango and shrimp quesadilla . my friend ordered nachos . the food was not good . i and my friend could not finish our food and we had stomach aches immediately . Table 1: Saliency maps of BERT generated by two common gradient methods and by our proposed method, ALTI, for a negative sentiment prediction example of …

Content truncated to 500 words