Characterizing Verbatim Short-Term Memory in Neural Language Models
41628Authors
nan
Abstract
AbstractWhen a language model is trained to predict natural language sequences, its prediction at each moment depends on a representation of prior context. What kind of information about the prior context can language models retrieve? We tested whether language models could retrieve the exact words that occurred previously in a text. In our paradigm, language models (transformers and an LSTM) processed English text in which a list of nouns occurred twice. We operationalized retrieval as the reduction in surprisal from the first to the second list. We found that the transformers retrieved both the identity and ordering of nouns from the first list. Further, the transformers’ retrieval was markedly enhanced when they were trained on a larger corpus and with greater model depth. Lastly, their ability to index prior tokens was dependent on learned attention patterns. In contrast, the LSTM exhibited less precise retrieval, which was limited to list-initial tokens and to short intervening texts. The LSTM’s retrieval was not sensitive to the order of nouns and it improved when the list was semantically coherent. We conclude that transformers implemented something akin to a working memory system that could flexibly retrieve individual token representations across arbitrary delays; conversely, the LSTM maintained a coarser and more rapidly-decaying semantic gist of prior tokens, weighted toward the earliest items.
Keywords
Full Content
Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL), pages 13 - 26 December 7-8, 2022 ©2022 Association for Computational Linguistics Collateral facilitation in humans and language models James A. Michaelov Department of Cognitive Science University of California, San Diego j1michae@ucsd.edu Benjamin K. Bergen Department of Cognitive Science University of California, San Diego bkbergen@ucsd.edu Abstract Are the predictions of humans and language models affected by similar things? Research suggests that while comprehending language, humans make predictions about upcoming words, with more predictable words being pro- cessed more easily. However, evidence also shows that humans display a similar processing advantage for highly anomalous words when these words are semantically related to the pre- ceding context or to the most probable contin- uation. Using stimuli from 3 psycholinguis- tic experiments, we find that this is also al- most always also the case for 8 contemporary transformer language models (BERT, ALBERT, RoBERTa, XLM-R, GPT-2, GPT-Neo, GPT-J, and XGLM). We then discuss the implications of this phenomenon for our understanding of both human language comprehension and the predictions made by language models. 1 Introduction Humans process words more easily when they more contextually predictable, whether predictabil- ity is determined by humans (Fischler and Bloom, 1979; Brothers and Kuperberg, 2021) or language models (McDonald and Shillcock, 2003; Levy, 2008; Smith and Levy, 2013). Work on the N400, a neural signal of processing difficulty, has pro- vided evidence that the neurocognitive system un- derlying human language comprehension preacti- vates words based on the extent to which they are predictable from the preceding context—thus, pre- dictable words are easier to process because they or their features have already been activated before they are encountered (Kutas and Hillyard, 1984; Van Petten and Luka, 2012). This has led many to argue that we should consider the human language comprehension system to be engaging in prediction (DeLong et al., 2005; Kutas et al., 2011; Van Pet- ten and Luka, 2012; Bornkessel-Schlesewsky and Schlesewsky, 2019; Kuperberg et al., 2020; De- Long and Kutas, 2020; Brothers and Kuperberg, 2021). However, words that are either semantically re- lated to the elements of the preceding context or to the most likely next word are also processed more easily, even if they are semantically implausible and ostensibly unpredictable. These are known as related anomaly effects. For an example of the for- mer, consider the sentences in (1) that were used as experimental stimuli by Metusalem et al. (2012). (1) My friend Mike went mountain biking recently. He lost control for a moment and ran right into a tree. It’s a good thing he was wearing his ______. (a) helmet (b) dirt (c) table Helmet is the most predictable continuation of the sentence, as determined based on cloze proba- bility (Taylor, 1953, 1957)—the proportion of peo- ple to fill in a gap in a sentence with a specific word. Thus, unsurprisingly, helmet elicited the smallest N400 response, indicating that it is most easily processed. Dirt and table are both implau- sible continuations, and equally …
Content truncated to 500 words
Labeled Text
AbstractWhen a language model is trained to predict natural language sequences, its prediction at each moment depends on a representation of prior context. What kind of information about the prior context can language models retrieve? We tested whether language models could retrieve the exact words that occurred previously in a text. In our paradigm, language models (transformers and an LSTM) processed English text in which a list of nouns occurred twice. We operationalized retrieval as the reduction in surprisal from the first to the second list. We found that the transformers retrieved both the identity and ordering of nouns from the first list. Further, the transformers’ retrieval was markedly enhanced when they were trained on a larger corpus and with greater model depth. Lastly, their ability to index prior tokens was dependent on learned attention patterns. In contrast, the LSTM exhibited less precise retrieval, which was limited to list-initial tokens and to short intervening texts. The LSTM’s retrieval was not sensitive to the order of nouns and it improved when the list was semantically coherent. We conclude that transformers implemented something akin to a working memory system that could flexibly retrieve individual token representations across arbitrary delays; conversely, the LSTM maintained a coarser and more rapidly-decaying semantic gist of prior tokens, weighted toward the earliest items.
Publication Details
- Paper ID:
- 41628
- Year:
- 2022
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan