PreCog: Exploring the Relation between Memorization and Performance in Pre-trained Language Models

Proceedings of Recent Advances in Natural Language Processing, pages 961–967 Varna, Sep 4–6, 2023 https://doi.org/10.26615/978-954-452-092-2_103 961 PreCog: Exploring the Relation between Memorization and Performance in Pre-trained Language Models Leonardo Ranaldi (∗,•), Elena Sofia Ruzzetti(∗), Fabio Massimo Zanzotto(∗) (•) Idiap Research Institute, Martigny, Switzerland (∗) ART Group, Department of Enterprise Engineering, University of Rome Tor Vergata, Rome, Italy [first name].[last name]@uniroma2.it Abstract Large Language Models (LLMs) are impres- sive machines with the ability to memorize, possibly generalized learning examples. We present here a small, focused contribution to the analysis of the interplay between memoriza- tion and performance of BERT in downstream tasks. We propose PreCog, a measure for eval- uating memorization from pre-training, and we analyze its correlation with the BERT’s per- formance. Our experiments show that highly memorized examples are better classified, sug- gesting memorization is an essential key to suc- cess for BERT1. 1 Introduction Large Language Models (LLMs) (Brown et al., 2020; Touvron et al., 2023) are intriguing machines dominating the arena of NLP tasks with their abil- ity to memorize generalizations of texts in syn- thetic neurons. After long pre-training on large amounts of unlabeled data, LLMs have been shown to learn effectively downstream tasks with limited labeled data (Howard and Ruder, 2018) and gen- eralize in out-of-distribution examples (Hendrycks et al., 2020). Extensive studies have shown that these models tend to mimic traditional linguistic syntactic models (McCoy et al., 2019; Ranaldi and Pucci, 2023) and traditional NLP. Hence, a crucial issue is to clarify why PLTMs exploit pre-training better than traditional NLP modules exploit anno- tated corpora. Understanding the learning process of LLMs may help in understanding their results in down- stream tasks and in improving their linguistic rep- resentations in scenarios where they fail (Kumar et al., 2020). Indeed, unlike traditional general NLP modules in pipelines, LLMs need to be fine-tuned 1The code and is publicly available at: https://github. com/ART-Group-it/PreCog for the specific tasks (Devlin et al., 2019) and, even- tually, domain-adapted on the specific language of the novel corpus (Jin et al., 2022). Moreover, as with many other machine learning models, fine- tuned PTLMs lose their ability to solve a task if subsequently fine-tuned to another task (Xu et al., 2020) although they apparently do not change their language models (Merchant et al., 2020). This phe- nomenon is known as catastrophic forgetting (Kirk- patrick et al., 2017) in machine learning. Then, it is still unclear how these models exploit pre-training and training examples. LLMs, such as BERT (Devlin et al., 2019), have shown to have an impressive ability to memorize and possibly generalize learning examples. This ability has been largely investigated as it may be ex- tremely harmful. In fact, these models may reveal sensitive information that has been acquired dur- ing pre-training. For example, memories of GPTs (Radford and Narasimhan, 2018) have been vio- lated and produced phone numbers, and usernames (Carlini et al., 2021; Thakkar et al., 2021). How- ever, this simple ability to memorize may play a crucial role in the …

Content truncated to 500 words