Interpreting Pretrained Contextualized Representations via Reductions to Static Embeddings

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4758–4781 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 4758 Interpreting Pretrained Contextualized Representations via Reductions to Static Embeddings Rishi Bommasani Cornell University rb724@cornell.edu Kelly Davis Mozilla Corporation kdavis@mozilla.com Claire Cardie Cornell University cardie@cs.cornell.edu Abstract Contextualized representations (e.g. ELMo, BERT) have become the default pretrained representations for downstream NLP applica- tions. In some settings, this transition has ren- dered their static embedding predecessors (e.g. Word2Vec, GloVe) obsolete. As a side-effect, we observe that older interpretability methods for static embeddings — while more mature than those available for their dynamic counter- parts — are underutilized in studying newer contextualized representations. Consequently, we introduce simple and fully general meth- ods for converting from contextualized repre- sentations to static lookup-table embeddings which we apply to 5 popular pretrained mod- els and 9 sets of pretrained weights. Our anal- ysis of the resulting static embeddings notably reveals that pooling over many contexts signif- icantly improves representational quality un- der intrinsic evaluation. Complementary to an- alyzing representational quality, we consider social biases encoded in pretrained represen- tations with respect to gender, race/ethnicity, and religion and find that bias is encoded dis- parately across pretrained models and internal layers even for models that share the same training data. Concerningly, we find dramatic inconsistencies between social bias estimators for word embeddings. 1 Introduction Word embeddings (Bengio et al., 2003; Collobert and Weston, 2008; Collobert et al., 2011) have been a hallmark of modern natural language processing (NLP) for many years. Embedding methods have been broadly applied and have experienced parallel and complementary innovations alongside neural network methods for NLP. Advances in embed- ding quality in part have come from integrating additional information such as syntax (Levy and Goldberg, 2014a; Li et al., 2017), morphology (Cot- terell and Sch¨utze, 2015), subwords (Bojanowski et al., 2017), subcharacters (Stratos, 2017; Yu et al., 2017) and, most recently, context (Peters et al., 2018; Devlin et al., 2019). Due to their tremendous representational power, pretrained contextualized representations, in particular, have seen widespread adoption across myriad subareas of NLP. The recent dominance of pretrained contextual- ized representations such as ELMo (Peters et al., 2018) and BERT (Devlin et al., 2019) has served as the impetus for exciting and diverse interpretability research: Liu et al. (2019a); Tenney et al. (2019a) study what is learned across the layers of these models, Tenney et al. (2019b); Ethayarajh (2019) consider what is learned from context, Clark et al. (2019); Michel et al. (2019) look at specific atten- tion heads, Hewitt and Manning (2019); Ettinger (2020) address linguistic understanding such as syn- tax and negation, and Wallace et al. (2019); Tan and Celis (2019) address ethical concerns such as secu- rity (adversarial robustness) and social bias. In fact, the neologism BERTology was coined specifically to describe this flurry of interpretability research.1 While these works have provided nuanced fine- grained analyses by creating new interpretability schema/techniques, we instead take an alternate ap- proach of …

Content truncated to 500 words