Inspecting the concept knowledge graph encoded by modern language models

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2984–3000 August 1–6, 2021. ©2021 Association for Computational Linguistics 2984 Inspecting the concept knowledge graph encoded by modern language models Carlos Aspillaga1, Marcelo Mendoza2, Alvaro Soto1 1 Computer Science Department, Pontificia Universidad Cat´olica de Chile 2 Department of Informatics, Universidad T´ecnica Federico Santa Mar´ıa, Chile cjaspill@uc.cl,mmendoza@inf.utfsm.cl,asoto@ing.puc.cl Abstract The field of natural language understanding has experienced exponential progress in the last few years, with impressive results in sev- eral tasks. This success has motivated re- searchers to study the underlying knowledge encoded by these models. Despite this, at- tempts to understand their semantic capabil- ities have not been successful, often lead- ing to non-conclusive, or contradictory con- clusions among different works. Via a prob- ing classifier, we extract the underlying knowl- edge graph of nine of the most influential lan- guage models of the last years, including word embeddings, text generators, and context en- coders. This probe is based on concept relat- edness, grounded on WordNet. Our results re- veal that all the models encode this knowledge, but suffer from several inaccuracies. Further- more, we show that the different architectures and training strategies lead to different model biases. We conduct a systematic evaluation to discover specific factors that explain why some concepts are challenging. We hope our insights will motivate the development of mod- els that capture concepts more precisely. 1 Introduction Natural language processing (NLP) encompasses a wide variety of applications such as summarization (Kovaleva et al., 2019), information retrieval (Zhan et al., 2020), and machine translation (Tang et al., 2018), among others. Currently, the use of pre- trained language models has become the de facto starting point to tackle most of these tasks. The usual pipeline consists of finetuning a pre-trained language model by using a discriminative learning objective to adapt the model to the requirements of each task. As key ingredients, these models are pre- trained using massive amounts of unlabeled data that can include millions of documents and billions of parameters. Massive data and parameters are supplemented with a suitable learning architecture, resulting in a highly powerful but also complex model whose internal operation is hard to analyze. The success of pre-trained language models has driven the interest to understand the mechanisms they use to solve NLP tasks. As an example, in the case of BERT (Devlin et al., 2019), one of the most popular pre-trained models based on the Transformer (Vaswani et al., 2017), several studies have attempted to access the knowledge encoded in its layers and attention heads (Tenney et al., 2019b; Devlin et al., 2019; Hewitt and Manning, 2019). In particular, Jawahar et al. (2019) shows that BERT can solve tasks at a syntactic level by using Trans- former blocks to encode a soft hierarchy of features at different levels of abstraction. Similarly, Hewitt and Manning (2019) show that BERT is capable of encoding structural information from text. In particular, using a structural probe, they show that syntax trees are embedded in a linear …

Content truncated to 500 words