Do language models have coherent mental models of everyday things?

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 1892–1913 July 9-14, 2023 ©2023 Association for Computational Linguistics Do language models have coherent mental models of everyday things? Yuling Gu and Bhavana Dalvi Mishra and Peter Clark Allen Institute for AI, Seattle, WA {yulingg,bhavanad,peterc}@allenai.org Abstract When people think of everyday things like an egg, they typically have a mental image asso- ciated with it. This allows them to correctly judge, for example, that “the yolk surrounds the shell” is a false statement. Do language models similarly have a coherent picture of such ev- eryday things? To investigate this, we propose a benchmark dataset consisting of 100 every- day things, their parts, and the relationships between these parts, expressed as 11,720 “X relation Y?” true/false questions. Using these questions as probes, we observe that state-of- the-art pre-trained language models (LMs) like GPT-3 and Macaw have fragments of knowl- edge about these everyday things, but do not have fully coherent “parts mental models” (54- 59% accurate, 19-43% conditional constraint violation). We propose an extension where we add a constraint satisfaction layer on top of the LM’s raw predictions to apply common- sense constraints. As well as removing incon- sistencies, we find that this also significantly improves accuracy (by 16-20%), suggesting how the incoherence of the LM’s pictures of everyday things can be significantly reduced.1 1 Introduction Psychologists and cognitive scientists hypothesize that humans develop mental models of the world, namely internal, conceptual representations of the environment which we base our decisions and ac- tions on (Ha and Schmidhuber, 2018; Jonassen and Henning, 1996). Hespos and Spelke (2004) observed that 5-month-old human infants exhibit understanding of mechanical properties of objects in terms of arrangements and motions of surfaces, well before they can understand language. Draw- ing loosely on this idea, but without making any claims about how LMs reason internally (Shanahan, 1We make our data and code publicly available at https: //github.com/allenai/everyday-things. Figure 1: While humans appear to have coherent men- tal pictures of everyday things (e.g., an egg, A), our question-asking probes suggest that LMs do not (e.g., one LM answered that the egg white both surrounds and is surrounded by the shell, B). This model incoherence can be reduced by applying commonsense constraints (e.g., surrounds is asymmetric), resulting in a more co- herent parts model (C). 2022; Andreas, 2022), we investigate if pre-trained language models show evidence of coherent inter- nal representations of everyday things, analogous to human mental models, via probing. We focus on mental models in the context of ordinary ob- jects that we encounter in our everyday lives. Such commonsense knowledge helps us understand how these everyday things work and how to interact with them. For example, when someone tries to make a fried egg, they know that it has a shell and 1892 that it can be cracked open to reveal the egg white and yolk inside. However, if a system does not have a coherent picture of such …

Content truncated to 500 words