What Does BERT Learn about the Structure of Language?

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3651–3657 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3651 What does BERT learn about the structure of language? Ganesh Jawahar Benoˆıt Sagot Djam´e Seddah Inria, France {firstname.lastname}@inria.fr Abstract BERT is a recent language representation model that has surprisingly performed well in diverse language understanding benchmarks. This result indicates the possibility that BERT networks capture structural information about language. In this work, we provide novel sup- port for this claim by performing a series of experiments to unpack the elements of English language structure learned by BERT. We first show that BERT’s phrasal representation cap- tures phrase-level information in the lower lay- ers. We also show that BERT’s intermediate layers encode a rich hierarchy of linguistic in- formation, with surface features at the bottom, syntactic features in the middle and semantic features at the top. BERT turns out to require deeper layers when long-distance dependency information is required, e.g. to track subject- verb agreement. Finally, we show that BERT representations capture linguistic information in a compositional way that mimics classical, tree-like structures. 1 Introduction BERT (Bidirectional Encoder Representations from Transformers) (Devlin et al., 2018) is a bidirectional variant of Transformer networks (Vaswani et al., 2017) trained to jointly predict a masked word from its context and to classify whether two sentences are consecutive or not. The trained model can be fine-tuned for downstream NLP tasks such as question answering and lan- guage inference without substantial modification. BERT outperforms previous state-of-the-art mod- els in the eleven NLP tasks in the GLUE bench- mark (Wang et al., 2018) by a significant margin. This remarkable result suggests that BERT could “learn” structural information about language. Can we unveil the representations learned by BERT to proto-linguistics structures? Answering this question could not only help us understand the reason behind the success of BERT but also its limitations, in turn guiding the design of improved architectures. This question falls under the topic of the interpretability of neural networks, a growing field in NLP (Belinkov and Glass, 2019). An im- portant step forward in this direction is Goldberg (2019), which shows that BERT captures syntac- tic phenomena well when evaluated on its ability to track subject-verb agreement. In this work, we perform a series of experiments to probe the nature of the representations learned by different layers of BERT. 1 We first show that the lower layers capture phrase-level information, which gets diluted in the upper layers. Second, we propose to use the probing tasks defined in Con- neau et al. (2018) to show that BERT captures a rich hierarchy of linguistic information, with sur- face features in lower layers, syntactic features in middle layers and semantic features in higher lay- ers. Third, we test the ability of BERT representa- tions to track subject-verb agreement and find that BERT requires deeper layers for handling harder cases involving long-distance dependencies. Fi- nally, we propose to use the recently introduced …

Content truncated to 500 words