Entity Tracking in Language Models

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 3835–3855 July 9-14, 2023 ©2023 Association for Computational Linguistics Entity Tracking in Language Models Najoung Kim∗ Department of Linguistics Boston University najoung@bu.edu Sebastian Schuster∗ Dept. of Language Science and Technology Saarland University seschust@lst.uni-saarland.de Abstract Keeping track of how states of entities change as a text or dialog unfolds is a key prereq- uisite to discourse understanding. Yet, there have been few systematic investigations into the ability of large language models (LLMs) to track discourse entities. In this work, we present a task probing to what extent a lan- guage model can infer the final state of an entity given an English description of the ini- tial state and a series of state-changing op- erations. We use this task to first investi- gate whether Flan-T5, GPT-3 and GPT-3.5 can track the state of entities, and find that only GPT-3.5 models, which have been pretrained on large amounts of code, exhibit this abil- ity. We then investigate whether smaller mod- els pretrained primarily on text can learn to track entities, through finetuning T5 on sev- eral training/evaluation splits. While perfor- mance degrades for more complex splits, we find that even when evaluated on a different set of entities from training or longer operation se- quences, a finetuned model can perform non- trivial entity tracking. Taken together, these results suggest that language models can learn to track entities but pretraining on text corpora alone does not make this capacity surface. 1 Introduction A key prerequisite to long-context understanding and generating coherent text is the ability to accu- rately represent entities as the discourse unfolds (Karttunen, 1976; Groenendijk and Stokhof, 1991; Heim, 2002; Nieuwland and Van Berkum, 2006; Kamp et al., 2011, i.a.). For example, consider the following example in the context of a recipe: (1) Put the eggs, sugar, flour, and baking pow- der in a bowl and mix to form a light batter. Make sure that the final batter does not con- tain any lumps of flour or sugar. ∗Equal contribution. Author order was determined by a fair process of letting Cookie the cat pick a colored ball. Q: Box 1 contains the book. Box 2 contains the apple. Box 4 contains the brain. Move the book into Box 2. Put the bell into Box 4. Move the bell and the brain into Box 5. Box 2 contains ____ A: the apple and the book Figure 1: A sketch of our entity tracking task. In order to understand this instruction, several dis- tinct abilities are necessary: New discourse entity recognition: recognizing when new discourse entities are introduced. E.g., a bowl introduces a new discourse entity but the final batter or any lumps of ... does not. Coreference resolution: associating referring ex- pressions with discourse entities. E.g., a light bat- ter and the final batter refer to the same entity. Discourse entity tracking: tracking the state changes made to each discourse entity. E.g., the eggs are …

Content truncated to 500 words