Understanding Learning Dynamics Of Language Models with SVCCA

Proceedings of NAACL-HLT 2019, pages 3257–3267 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 3257 Understanding Learning Dynamics Of Language Models with SVCCA Naomi Saphra and Adam Lopez Institute for Language, Cognition, and Computation University of Edinburgh n.saphra@ed.ac.uk alopez@ed.ac.uk Abstract Research has shown that neural models im- plicitly encode linguistic features, but there has been no research showing how these en- codings arise as the models are trained. We present the first study on the learning dy- namics of neural language models, using a simple and flexible analysis method called Singular Vector Canonical Correlation Anal- ysis (SVCCA), which enables us to compare learned representations across time and across models, without the need to evaluate directly on annotated data. We probe the evolution of syntactic, semantic, and topic representations and find that part-of-speech is learned earlier than topic; that recurrent layers become more similar to those of a tagger during training; and embedding layers less similar. Our results and methods could inform better learning al- gorithms for NLP models, possibly to incor- porate linguistic information more effectively. 1 Introduction Large neural networks have a notorious capacity to memorize training data (Zhang et al., 2016), but their high accuracy on many NLP tasks shows that they nonetheless generalize. One apparent ex- planation for their performance is that they learn linguistic generalizations even without explicit su- pervision for those generalizations—for example, that subject and verb number agree in English (Linzen et al., 2016); that derivational suffixes at- tach to only specific parts of speech (Kementched- jhieva and Lopez, 2018); and that short segments of speech form natural clusters corresponding to phonemes (Alishahi et al., 2017). These studies tell us that neural models learn to implicitly rep- resent linguistic categories and their interactions. But how do they learn these representations? One clue comes from the inspection of multi- layer models, which seem to encode lexical cate- gories in lower layers, and more contextual cate- gories in higher layers. For example, Blevins et al. (2018) found that a word’s part of speech (POS) is encoded by lower layers, and the POS of its syn- tactic parent is encoded by higher layers; while Belinkov et al. (2018) found that POS is encoded by lower layers and semantic category is encoded by higher layers. More generally, the most useful layer for an arbitrary NLP task seems to depend on how “high-level” the task is (Peters et al., 2018). Since we know that lower layers in a multi-layer model converge to their final representations more quickly than higher layers (Raghu et al., 2017), it is likely that models learn local lexical categories like POS earlier than they learn higher-level lin- guistic categories like semantic class. How and when do neural representations come to encode specific linguistic categories? Answers could explain why neural models work and help us improve learning algorithms. We investigate how representations of linguistic structure are learned over time in neural language models (LMs), which are central to NLP: on their …

Content truncated to 500 words