How Do Neural Sequence Models Generalize? Local and Global Cues for Out-of-Distribution Prediction

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5513–5526 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 5513 How Do Neural Sequence Models Generalize? Local and Global Context Cues for Out-of-Distribution Prediction Anthony Bau and Jacob Andreas Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology {abau, jda}@mit.edu Abstract After a neural sequence model encounters an unexpected token, can its behavior be pre- dicted? We show that RNN and transformer language models exhibit structured, consistent generalization in out-of-distribution contexts. We begin by introducing two idealized mod- els of generalization in next-word prediction: a local context model in which generalization is consistent with the last word observed, and a global context model in which generaliza- tion is consistent with the global structure of the input. In experiments in English, Finnish, Mandarin, and random regular languages, we demonstrate that neural language models inter- polate between these two forms of generaliza- tion: their predictions are well-approximated by a log-linear combination of local and global predictive distributions. We then show that, in some languages, noise mediates the two forms of generalization: noise applied to input to- kens encourages global generalization, while noise in history representations encourages lo- cal generalization. Finally, we offer a prelimi- nary theoretical explanation of these results by proving that the observed interpolation behav- ior is expected in log-linear models with a par- ticular feature correlation structure. These re- sults help explain the effectiveness of two pop- ular regularization schemes and show that as- pects of sequence model generalization can be understood and controlled. 1 Introduction Neural language models (LMs) play a key role in language processing systems for tasks as diverse as machine translation, dialogue, and automated speech recognition (Baziotis et al., 2020; Sordoni et al., 2015; Mikolov et al., 2010). These LMs, which model distributions over words in context via recurrent, convolutional, or attentional neural networks, have been found to consistently outper- form finite-state approaches to language modeling based on hidden Markov models (Kuhn et al., 1994) Neural Language Model Global context XG Local ctx. XL Next token pLM(Xn)? the pizza ate cat tasted Corpus data p(X1:n) Regular language p(X1:n) A C B w x y A C x ? x w y x? y? w? the? delicious? tasted? Global generalization: pLM(Xn) ≈ p(Xn ∣ XG) Local generalization: pLM(Xn) ≈ p(Xn ∣ XL) Local context model Global context model Interpolation Figure 1: We develop formal models of the predic- tions of neural language models in surprising contexts in which local information (e.g. the most recent token) and global information (e.g. the rest of the sentence) conflict (top). In these out-of-distribution contexts, pre- dictors trained on both synthetic and natural languages favor either local or global information, but are best approximated by an interpolation of a local-only and global-only predictor (bottom). or n-gram statistics (Miller and Selfridge, 1950). But improved predictive power comes at the cost of increased model complexity and a loss of trans- parency. While it is possible to characterize (and even …

Content truncated to 500 words