Запоминание
Related Papers
Are Pretrained Language Models Symbolic Reasoners over Knowledge?
AbstractHow can pretrained language models (PLMs) learn factual knowledge from the training set? We investigate the two most important mechanisms: reasoning and memorization. Prior work has attempted to quantify the …...
Investigating Memorization of Conspiracy Theories in Text Generation
The adoption of natural language generation (NLG) models can leave individuals vulnera- ble to the generation of harmful information memorized by the models, such as conspiracy theories. While previous studies …...
Quantifying Adaptability in Pre-trained Language Models with 500 Tasks
AbstractWhen a neural language model (LM) is adapted to perform a new task, what aspects of the task predict the eventual performance of the model? In NLP, systematic features of …...
PreCog: Exploring the Relation between Memorization and Performance in Pre-trained Language Models
AbstractLarge Language Models (LLMs) are impressive machines with the ability to memorize, possibly generalized learning examples. We present here a small, focused contribution to the analysis of the interplay between …...