Who’s on First?: Probing the Learning and Representation Capabilities of Language Models on Deterministic Closed Domains

Proceedings of the 25th Conference on Computational Natural Language Learning (CoNLL), pages 210–222 November 10–11, 2021. ©2021 Association for Computational Linguistics 210 Who’s on First?: Probing the Learning and Representation Capabilities of Language Models on Deterministic Closed Domains David Demeter Northwestern University ddemeter@u.northwestern.edu Doug Downey Allen Institute for AI dougd@allenai.org Abstract The capabilities of today’s natural language processing systems are typically evaluated us- ing large datasets of curated questions and an- swers. While these are critical benchmarks of progress, they also suffer from weakness due to artificial distributions and incomplete knowledge. Artifacts arising from artificial dis- tributions can overstate language model per- formance, while incomplete knowledge limits fine-grained analysis. In this work, we introduce a complementary benchmarking approach based on SimPlified Language Activity Traces (SPLAT). SPLATs are corpora of language encodings of activ- ity in some closed domain (we study traces from chess and baseball games in this work). SPLAT datasets use naturally-arising distribu- tions, allow the generation of question-answer pairs at scale, and afford complete knowledge in their closed domains. We show that lan- guage models of three different architectures can answer questions about world states using only verb-like encodings of activity. Our ap- proach is extensible to new language models and additional question-answering tasks. 1 Introduction Language models (LMs) are pre-trained on running text and have been shown to have a remarkable ability to transform this raw observational data into models that capture syntax, semantics, and certain forms of world knowledge such as common sense (Tenney et al., 2019; Rogers et al., 2020). However, accurately measuring the models’ representation and learning capabilities can be difficult because the tasks that they are evaluated on can be sub- jective, require expensive annotation, and suffer from dataset artifacts that allow models to achieve spurious high performance. In this paper, we propose novel, complemen- tary benchmarks for evaluating the potential of Figure 1: Baseball Illustration. A typical presentation of baseball is a snapshot of the scoreboard and field (a). We represent these as a sequence of world-states and encode gameplay as a sequence of SPLAT tokens that map to deterministic changes in world-states (b). SPLAT encoding makes it possible to pose question- answer pairs with complete knowledge in a language modeling setting, where a single indicator represents each question (c). different language model architectures. Our ap- proach involves pre-training and evaluating mod- els on what we term SimPlified Language Activity Traces (SPLAT): corpora of language encodings of activity in a closed domain. SPLATs have three characteristics that make them valuable for probing the capabilities of LMs: (a) they are comprised of data from existing, organic distributions not de- signed by the experimenters, helping to prevent spurious artifacts, (b) they can be acquired at scale, enabling the construction of well-trained models, 211 and (c) they admit automatic construction of ques- tions with known, objectively correct answers, that probe the linguistic or cognitive potential of the models. We introduce two SPLAT datasets based on language-like traces of chess and baseball games (see Figure 1). …

Content truncated to 500 words