“Average” Approximates “First Principal Component”? An Empirical Analysis on Representations from Neural Language Models

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5594–5603 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 5594 “Average” Approximates “First Principal Component”? An Empirical Analysis on Representations from Neural Language Models Zihan Wang 1 Chengyu Dong 1 Jingbo Shang ∗,1,2 1 Department of Computer Science and Engineering, University of California San Diego, CA, USA 2 Halıcıo˘glu Data Science Institute, University of California San Diego, CA, USA {ziw224, cdong, jshang}@ucsd.edu ∗ Abstract Contextualized representations based on neu- ral language models have furthered the state of the art in various NLP tasks. Despite its great success, the nature of such repre- sentations remains a mystery. In this pa- per, we present an empirical property of these representations—“average” ≈ “first principal component”. Specifically, experiments show that the average of these representations shares almost the same direction as the first princi- pal component of the matrix whose columns are these representations. We believe this ex- plains why the average representation is al- ways a simple yet strong baseline. Our fur- ther examinations show that this property also holds in more challenging scenarios, for exam- ple, when the representations are from a model right after its random initialization. Therefore, we conjecture that this property is intrinsic to the distribution of representations and not nec- essarily related to the input structure. We re- alize that these representations empirically fol- low a normal distribution for each dimension, and by assuming this is true, we demonstrate that the empirical property can be in fact de- rived mathematically. 1 Introduction A large variety of state-of-the-art methods in NLP tasks nowadays are built upon contextualized rep- resentations from pre-trained neural language mod- els, such as ELMo (Peters et al., 2018), BERT (De- vlin et al., 2019), and XLNet (Yang et al., 2019). Despite the great success, we lack understandings about the nature of such representations. For exam- ple, Aharoni and Goldberg (2020) have shown that averaging BERT representations in a sentence can preserve its domain information. However, to our best knowledge, there is no analysis on what leads to the power of averaging representations. ∗Jingbo Shang is the corresponding author. Input : Average '(!×# First Principal Component )!×# cos (, ) =0.999 … Pre-trained Neural Language Model … … … … … ($ … (% (& (' … … Matrix 2(×) ) = 24# = "! 0 0 "" 0 0 2(×) 5(×( 6(×) 7)×) * 4# = "! 0 0 "" 0 0 !!×# "!×! #!×# $#×# $ %% Figure 1: Visualization of our discovered empirical property: “average” ≈ “first principal component”. Table 1: Average and minimum absolute cosine similar- ity of last layer representations between r and p from 4,000 tests. As a reference, ri drawn from a uniformly random distribution would lead to Average of .0149. Model AG’s news KP20k Dbpedia Average Min Average Min Average Min BERT .9994 .9908 .9995 .9958 .9988 .9845 RoBERTa .9989 .9984 .9990 .9982 .9987 .9980 XLNet .9990 .9874 .9991 .9932 .9994 .9856 ELMo .9957 .9681 …

Content truncated to 500 words