Effects of Parameter Norm Growth During Transformer Training: Inductive Bias from Gradient Descent

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1766–1781 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 1766 Effects of Parameter Norm Growth During Transformer Training: Inductive Bias from Gradient Descent William Merrill∗† Vivek Ramanujan∗‡ Yoav Goldberg∗§ Roy Schwartz¶ Noah A. Smith∗‡ ∗ Allen Institute for AI † New York University ‡ University of Washington § Bar Ilan University ¶Hebrew University of Jerusalem willm@nyu.edu ramanv@cs.washington.edu {yoavg,noah}@allenai.org roys@cs.huji.ac.il Abstract The capacity of neural networks like the widely adopted transformer is known to be very high. Evidence is emerging that they learn successfully due to inductive bias in the training routine, typically a variant of gradi- ent descent (GD). To better understand this bias, we study the tendency for transformer pa- rameters to grow in magnitude (ℓ2 norm) dur- ing training, and its implications for the emer- gent representations within self attention lay- ers. Empirically, we document norm growth in the training of transformer language models, including T5 during its pretraining. As the pa- rameters grow in magnitude, we prove that the network approximates a discretized network with saturated activation functions. Such “sat- urated” networks are known to have a reduced capacity compared to the full network family that can be described in terms of formal lan- guages and automata. Our results suggest satu- ration is a new characterization of an inductive bias implicit in GD of particular interest for NLP. We leverage the emergent discrete struc- ture in a saturated transformer to analyze the role of different attention heads, finding that some focus locally on a small number of po- sitions, while other heads compute global av- erages, allowing counting. We believe under- standing the interplay between these two capa- bilities may shed further light on the structure of computation within large transformers. 1 Introduction Transformer-based models (Vaswani et al., 2017) like BERT (Devlin et al., 2019), XLNet (Yang et al., 2019), RoBERTa (Liu et al., 2019), and T5 (Raffel et al., 2019) have pushed the state of the art on an impressive array of NLP tasks. Overparameterized transformers are known to be unversal approxima- tors (Yun et al., 2020), suggesting their general- ization performance ought to rely on useful biases or constraints imposed by the learning algorithm. Despite various attempts to study these biases in transformers (Rogers et al., 2020; Lovering et al., 2021), it remains an interesting open question what they are, or even how to characterize them in a way relevant to the domain of language. In this work, we take the perspective that thor- oughly understanding the dynamics of gradient de- scent (GD) might clarify the linguistic biases of transformers, and the types of representations they acquire. We start by making a potentially surprising empirical observation (§3): the parameter ℓ2 norm grows proportional to √ t (where t is the timestep) during the training of T5 (Raffel et al., 2019) and other transformers. We refer to the phenomenon of growing parameter norm during training as norm growth. Previous work has analyzed norm …

Content truncated to 500 words