Roles and Utilization of Attention Heads in Transformer-based Neural Language Models

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3404–3417 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 3404 Roles and Utilization of Attention Heads in Transformer-based Neural Language Models Jae-young Jo School of Computing, KAIST Dingbro AI Research kevin.jo@dingbro.ai Sung-hyon Myaeng School of Computing, KAIST myaeng@kaist.ac.kr Abstract Sentence encoders based on the transformer architecture have shown promising results on various natural language tasks. The main im- petus lies in the pre-trained neural language models that capture long-range dependencies among words, owing to multi-head attention that is unique in the architecture. However, little is known for how linguistic properties are processed, represented, and utilized for downstream tasks among hundreds of atten- tion heads inside the pre-trained transformer- based model. For the initial goal of examin- ing the roles of attention heads in handling a set of linguistic features, we conducted a set of experiments with ten probing tasks and three downstream tasks on four pre-trained transformer families (GPT, GPT2, BERT, and ELECTRA). Meaningful insights are shown through the lens of heat map visualization and utilized to propose a relatively simple sentence representation method that takes advantage of most influential attention heads, resulting in additional performance improvements on the downstream tasks. 1 Introduction Sentence encoders in transformer architectures as in GPT, BERT (Vaswani et al., 2017; Radford, 2018; Devlin et al., 2019) and ELECTRA (Clark et al., 2020) have shown promising results on vari- ous natural language understanding (NLU) tasks, such as question answering, text entailment and natural language inference (NLI) (Bowman et al., 2015), owing to their pre-training capabilities in modeling languages. The pre-training effects of the transformer-based approaches are known to be crucial for obtaining superior performance in various downstream NLU tasks. The main impetus lies in capturing long- range dependencies among words obtainable with bidirectional learning and self-attention (Devlin et al., 2019) and sufficiently varied corpora of a large quantity (Radford et al., 2019). Despite all the recent successes of the transformer-based models, little is known for how linguistic properties are processed and represented internally when the architectures are used. Given that self-attention heads are unique in the family of transformer architectures, we attempt to answer the question of how basic linguistic properties are captured with the attention heads across the models and used for downstream tasks. Once we figure out the roles of attention heads in “storing” various linguistic properties, we should be able to modulate them to maximize the performance of the down- stream tasks. Given the motivation, we analyze several pub- licly available pre-trained transformer encoders (BERT, GPT, GPT2, and ELECTRA) trained with different model capacities ranging from 144 to 384 attention heads and 12 to 24 layers. Considering the output vector from each attention head of an en- coder as a mini-sentence embedding, we examine whether certain linguistic properties are “stored” in embeddings among ten sentence probing tasks (Conneau and Kiela, 2018) that cover surface, syn- tactic, and semantic information and require differ- ent linguistic properties (e.g. the …

Content truncated to 500 words