The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 4396–4406, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 4396 The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives Elena Voita1,2 Rico Sennrich4,3 Ivan Titov3,2 1Yandex, Russia 2University of Amsterdam, Netherlands 3University of Edinburgh, Scotland 4University of Zurich, Switzerland lena-voita@yandex-team.ru sennrich@cl.uzh.ch ititov@inf.ed.ac.uk Abstract We seek to understand how the representations of individual tokens and the structure of the learned feature space evolve between layers in deep neural networks under different learning objectives. We focus on the Transformers for our analysis as they have been shown effec- tive on various tasks, including machine trans- lation (MT), standard left-to-right language models (LM) and masked language model- ing (MLM). Previous work used black-box probing tasks to show that the representations learned by the Transformer differ significantly depending on the objective. In this work, we use canonical correlation analysis and mutual information estimators to study how informa- tion flows across Transformer layers and how this process depends on the choice of learning objective. For example, as you go from bot- tom to top layers, information about the past in left-to-right language models gets vanished and predictions about the future get formed. In contrast, for MLM, representations initially acquire information about the context around the token, partially forgetting the token iden- tity and producing a more generalized token representation. The token identity then gets recreated at the top MLM layers. 1 Introduction Deep (i.e. multi-layered) neural networks have be- come the standard approach for many natural lan- guage processing (NLP) tasks, and their analysis has been an active topic of research. One popular approach for analyzing representations of neural models is to evaluate how informative they are for various linguistic tasks, so-called “probing tasks”. Previous work has made some interesting observa- tions regarding these representations; for example, Zhang and Bowman (2018) show that untrained LSTMs outperform trained ones on a word iden- tity prediction task; and Blevins et al. (2018) show that up to a certain layer performance of represen- tations obtained from a deep LM improves on a constituent labeling task, but then decreases, while with representations obtained from an MT encoder performance continues to improve up to the high- est layer. These observations have, however, been somewhat anecdotal and an explanation of the pro- cess behind such behavior has been lacking. In this paper, we attempt to explain more gen- erally why such behavior is observed. Rather than measuring the quality of representations obtained from a particular model on some auxiliary task, we characterize how the learning objective determines the information flow in the model. In particular, we consider how the representations of individual tokens in the Transformer evolve between layers under different learning objectives. We look at this task from the information bottle- neck perspective on learning in neural networks. Tishby and Zaslavsky (2015) state that “the goal …

Content truncated to 500 words