Revealing the Dark Secrets of BERT

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 4365–4374, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 4365 Revealing the Dark Secrets of BERT Olga Kovaleva, Alexey Romanov, Anna Rogers, Anna Rumshisky Department of Computer Science University of Massachusetts Lowell Lowell, MA 01854 {okovalev,arum,aromanov}@cs.uml.edu Abstract BERT-based architectures currently give state- of-the-art performance on many NLP tasks, but little is known about the exact mechanisms that contribute to its success. In the current work, we focus on the interpretation of self- attention, which is one of the fundamental un- derlying components of BERT. Using a sub- set of GLUE tasks and a set of handcrafted features-of-interest, we propose the methodol- ogy and carry out a qualitative and quantita- tive analysis of the information encoded by the individual BERT’s heads. Our findings sug- gest that there is a limited set of attention pat- terns that are repeated across different heads, indicating the overall model overparametriza- tion. While different heads consistently use the same attention patterns, they have varying impact on performance across different tasks. We show that manually disabling attention in certain heads leads to a performance improve- ment over the regular fine-tuned BERT mod- els. 1 Introduction Over the past year, models based on the Trans- former architecture (Vaswani et al., 2017) have become the de-facto standard for state-of-the-art performance on many natural language process- ing (NLP) tasks (Radford et al., 2018; Devlin et al., 2018). Their key feature is the self-attention mechanism that provides an alternative to conven- tionally used recurrent neural networks (RNN). One of the most popular Transformer-based models is BERT, which learns text representa- tions using a bi-directional Transformer encoder pre-trained on the language modeling task (De- vlin et al., 2018). BERT-based architectures have produced new state-of-the-art performance on a range of NLP tasks of different nature, do- main, and complexity, including question answer- ing, sequence tagging, sentiment analysis, and inference. State-of-the-art performance is usu- ally obtained by fine-tuning the pre-trained model on the specific task. In particular, BERT-based models are currently dominating the leaderboards for SQuAD1 (Rajpurkar et al., 2016) and GLUE benchmarks2 (Wang et al., 2018). However, the exact mechanisms that contribute to the BERT’s outstanding performance still re- main unclear. We address this problem through selecting a set of linguistic features of interest and conducting a series of experiments that aim to pro- vide insights about how well these features are captured by BERT. This paper makes the follow- ing contributions: • We propose a methodology and offer the first detailed analysis of BERT’s capacity to cap- ture different kinds of linguistic information by encoding it in its self-attention weights. • We present evidence of BERT’s over- parametrization and suggest a counter- intuitive yet frustratingly simple way of im- proving its performance, showing absolute gains of up to 3.2%. 2 Related work There have been several recent attempts to as- sess BERT’s ability to capture structural …

Content truncated to 500 words