Limits for learning with language models

Proceedings of the The 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023), pages 236–248 July 13-14, 2023 ©2023 Association for Computational Linguistics Limits for learning with Language Models Nicholas Asher CNRS, IRIT asher@irit.fr Swarnadeep Bhar Université de Toulouse 3 IRIT swarnadeep.bhar@irit.fr Akshay Chaturvedi Université de Toulouse 3 IRIT akshay91.isi@gmail.com Julie Hunter LINAGORA jhunter@linagora.com Soumya Paul TELINDUS soumya.paul@gmail.com Abstract With the advent of large language models (LLMs), the trend in NLP has been to train LLMs on vast amounts of data to solve diverse language understanding and generation tasks. The list of LLM successes is long and varied. Nevertheless, several recent papers provide em- pirical evidence that LLMs fail to capture im- portant aspects of linguistic meaning. Focusing on universal quantification, we provide a theo- retical foundation for these empirical findings by proving that LLMs cannot learn certain fun- damental semantic properties including seman- tic entailment and consistency as they are de- fined in formal semantics. More generally, we show that LLMs are unable to learn concepts beyond the first level of the Borel Hierarchy, which imposes severe limits on the ability of LMs, both large and small, to capture many aspects of linguistic meaning. This means that LLMs will continue to operate without formal guarantees on tasks that require entailments and deep linguistic understanding. 1 Introduction The success of large language models (LLMs) has led researchers in NLP to harness LLMs trained on vast amounts of data to solve a variety of language understanding and generation tasks, and some have claimed that LLMs can solve any task that can be specified via prompting (Brown et al., 2020). While the list of LLM successes is long, there have been several recent papers that provide empirical evi- dence that LLMs at least sometimes fail to capture important aspects of linguistic meaning (Kuhnle and Copestake, 2019; Sinha et al., 2020; Yuksek- gonul et al., 2022; Chaturvedi et al., 2022; Kalouli et al., 2022). Those who have dabbled in “BERTol- ogy” with respect to linguistic meaning often have the feeling that fixing one LLM deficiency just leads to the discovery of new ones. This paper provides a theoretical explanation of certain of these observed failings of LLMs. In particular, we prove that LLMs cannot learn the notions of semantic entailment or consistency as defined in formal semantics (Dowty et al., 1981) because they are incapable of mastering universal quantification. Our work builds on Siegelmann and Sontag (1992); Siegelmann (2012); Weiss et al. (2018), concerning the expressive power of neu- ral networks, but we focus on the learnability of semantic concepts and use novel tools. Our argument has widespread implications: not only does a general capacity to recognize seman- tic entailment and consistency underlie everyday conversational interactions, but the meanings of a great many common linguistic expressions depend on universal quantification. This set includes—but is certainly not limited to—a long list of quanti- fiers (every, some, many, most,... every other, ...), temporal adverbs (always, never, eventually) that are essential to planning (Lamport, 1980), …

Content truncated to 500 words