Too Much in Common: Shifting of Embeddings in Transformer Language Models and its Implications

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5117–5130 June 6–11, 2021. ©2021 Association for Computational Linguistics 5117 Too Much in Common: Shifting of Embeddings in Transformer Language Models and its Implications Daniel Bi´s Florida State University Tallahassee, USA bis@cs.fsu.edu Maksim Podkorytov Florida State University Tallahassee, USA maksim@cs.fsu.edu Xiuwen Liu Florida State University Tallahassee, USA liux@cs.fsu.edu Abstract The success of language models based on the Transformer architecture appears to be incon- sistent with observed anisotropic properties of representations learned by such models. We resolve this by showing, contrary to previous studies, that the representations do not occupy a narrow cone, but rather drift in common di- rections. At any training step, all of the em- beddings except for the ground-truth target em- bedding are updated with gradient in the same direction. Compounded over the training set, the embeddings drift and share common com- ponents, manifested in their shape in all the models we have empirically tested. Our ex- periments show that isotropy can be restored using a simple transformation.1 1 Introduction Word embeddings, both static (Mikolov et al., 2013a; Pennington et al., 2014) and contextual- ized (Peters et al., 2018), have been instrumental to the progress made in Natural Language Process- ing over the past decade (Turian et al., 2010; Wu et al., 2016; Liu et al., 2018; Peters et al., 2018; De- vlin et al., 2019). In recent years, language models based on Transformer architecture (Vaswani et al., 2017) have led to state-of-the-art performance on problems such as machine translation (Vaswani et al., 2017), question answering (Devlin et al., 2019; Liu et al., 2019b), and Word Sense Disam- biguation (Bevilacqua and Navigli, 2020), among others. However, it has been observed that repre- sentations from Transformers exhibit undesirable properties, such as anisotropy, that is tend to occupy only a small subspace of the embedding space. The observation has been documented by a number of studies (Gao et al., 2019; Ethayarajh, 2019; Wang et al., 2020). A similar property has been iden- tified in the past in static word embeddings (Mu 1The code and datasets used in this paper are available at https://github.com/danielbis/tooMuchInCommon. and Viswanath, 2018). To address the issues, post- processing methods (Mu and Viswanath, 2018), and regularization terms have been proposed (Gao et al., 2019; Wang et al., 2019c, 2020). However, the mechanism that leads to undesirable proper- ties remains unclear. Without understanding the mechanism, it is going to be difficult to address the fundamental issue properly. The deficiencies are most pronounced in the rep- resentations of rare words, as we will show in Section 4. Performance of pretrained language models is inconsistent and tends to decrease when input contains rare words (Schick and Schütze, 2020b,a). Schick and Schütze (2020a) observe that replacing a portion of words in the MNLI (Williams et al., 2018) entailment data set with less frequent synonyms leads to decrease in per- formance of BERT-base and RoBERTa-large by 30% and 21.8% respectively.2 After …

Content truncated to 500 words