Incorporating Residual and Normalization Layers into Analysis of Masked Language Models

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4547–4568 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 4547 Incorporating Residual and Normalization Layers into Analysis of Masked Language Models Goro Kobayashi1 Tatsuki Kuribayashi1,2 Sho Yokoi1,3 Kentaro Inui1,3 1 Tohoku University 2 Langsmith Inc. 3 RIKEN goro.koba@dc.tohoku.ac.jp {kuribayashi, yokoi, inui}@tohoku.ac.jp Abstract Transformer architecture has become ubiqui- tous in the natural language processing field. To interpret the Transformer-based models, their attention patterns have been extensively analyzed. However, the Transformer archi- tecture is not only composed of the multi- head attention; other components can also contribute to Transformers’ progressive perfor- mance. In this study, we extended the scope of the analysis of Transformers from solely the attention patterns to the whole attention block, i.e., multi-head attention, residual con- nection, and layer normalization. Our anal- ysis of Transformer-based masked language models shows that the token-to-token interac- tion performed via attention has less impact on the intermediate representations than pre- viously assumed. These results provide new intuitive explanations of existing reports; for example, discarding the learned attention pat- terns tends not to adversely affect the perfor- mance. The codes of our experiments are pub- licly available.1 1 Introduction Transformer architecture (Vaswani et al., 2017) has advanced the state of the art in a wide range of natural language processing (NLP) tasks (Devlin et al., 2019; Liu et al., 2019; Lan et al., 2020). Along with this, Transformers have become a ma- jor subject of research from the viewpoints of en- gineering (Rogers et al., 2020) and scientific stud- ies (Merkx and Frank, 2021; Manning et al., 2020). In particular, the multi-head attention, a core component of Transformers, has been extensively analyzed (Clark et al., 2019; Kovaleva et al., 2019; Reif et al., 2019; Lin et al., 2019; Mareˇcek and Rosa, 2019; Htut et al., 2019; Raganato and Tiede- mann, 2018; Tang et al., 2018). While these analy- ses suggest that the multi-head attention contributes 1https://github.com/gorokoba560/ norm-analysis-of-transformer (a) Existing analysis focusing only on the multi-head atten- tion (Kobayashi et al., 2020). (b) Proposed method incor- porating the whole attention block (i.e., multi-head atten- tion, residual connection, and layer normalization) into the analysis. Figure 1: Visualizations of the token-by-token inter- actions in each layer when a sentence pair is fed into the pre-trained BERT-base. The diagonal elements correspond to the effect of preserving the original in- put information. The contrast between Figures 1a and 1b demonstrates that the contextual information contributed less to the computation of the output rep- resentations than previously expected. to capturing linguistic information such as semantic and syntactic relations, some reports question the importance of attention. For example, several stud- ies in fields ranging from NLP (Michel et al., 2019; Kovaleva et al., 2019) to neuroscience (Toneva and Wehbe, 2019) empirically found that discarding learned attention patterns from Transformers re- tains or even improves their performance in down- stream tasks and the ability to simulate human brain activity. These observations imply that Transform- ers do not heavily rely …

Content truncated to 500 words