Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand?

Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand? William Merrill∗ Yoav Goldberg∗ † Roy Schwartz‡ Noah A. Smith∗§ ∗Allen Institute for AI, United States †Bar Ilan University, Israel ‡Hebrew University of Jerusalem, Israel §University of Washington, United States {willm,yoavg,roys,noah}@allenai.org Abstract Language models trained on billions of tokens have recently led to unprecedented results on many NLP tasks. This success raises the ques- tion of whether, in principle, a system can ever ‘‘understand’’ raw text without access to some form of grounding. We formally investigate the abilities of ungrounded systems to acquire meaning. Our analysis focuses on the role of ‘‘assertions’’: textual contexts that provide in- direct clues about the underlying semantics. We study whether assertions enable a system to emulate representations preserving semantic relations like equivalence. We find that asser- tions enable semantic emulation of languages that satisfy a strong notion of semantic trans- parency. However, for classes of languages where the same expression can take different values in different contexts, we show that em- ulation can become uncomputable. Finally, we discuss differences between our formal model and natural language, exploring how our re- sults generalize to a modal setting and other semantic relations. Together, our results sug- gest that assertions in code or language do not provide sufficient signal to fully emulate semantic representations. We formalize ways in which ungrounded language models appear to be fundamentally limited in their ability to ‘‘understand’’. 1 Introduction Recently, language models trained on huge data- sets of raw text have pushed the limits of natural language processing (Devlin et al., 2019; Raffel et al., 2019; Brown et al., 2020, among others). Such systems transcend the expert system para- digm, where rules about language and meaning are hardcoded into a system, as well as the supervised learning paradigm, where a notion of meaning is provided through ground-truth labels. Rather, analysis of massive language models has revealed that, to some degree, knowledge of syntactic and semantic dependencies can emerge without ex- plicit supervision (Rogers et al., 2020; Tenney et al., 2019). This knowledge can then be trans- ferred to a variety of downstream NLP tasks. Yet, today’s NLP systems built on large lan- guage models still fall short of human-level gen- eral understanding (Yogatama et al., 2019; Zhang et al., 2020). Brown et al. (2020) discuss the limi- tations of their GPT-3 language model compared with humans, suggesting that: Scaling up any LM-like model . . . may eventually run into (or could already be running into) the limits of the pretraining objective. This possibility raises an interesting theoretical question. What are the fundamental limits of learn- ing meaning from language modeling, even as- suming a perfect learner with access to unlimited data? Recently, Bender and Koller (2020) argued that achieving true natural language understanding from text alone is impossible, and that, to really get at meaning, some type of semantic grounding is necessary.1 Their style of argumentation largely focused on developing thought experiments, rather than making formal arguments. …

Content truncated to 500 words