oLMpics-On What Language Model Pre-training Captures

oLMpics-On What Language Model Pre-training Captures Alon Talmor1,2 Yanai Elazar1,3 Yoav Goldberg1,3 Jonathan Berant1,2 1The Allen Institute for AI 2Tel-Aviv University 3Bar-Ilan University {alontalmor@mail,joberant@cs}.tau.ac.il {yanaiela,yoav.goldberg}@gmail.com Abstract Recent success of pre-trained language models (LMs) has spurred widespread interest in the language capabilities that they possess. How- ever, efforts to understand whether LM repre- sentations are useful for symbolic reasoning tasks have been limited and scattered. In this work, we propose eight reasoning tasks, which conceptually require operations such as com- parison, conjunction, and composition. A fun- damental challenge is to understand whether the performance of a LM on a task should be attributed to the pre-trained representations or to the process of fine-tuning on the task data. To address this, we propose an eval- uation protocol that includes both zero-shot evaluation (no fine-tuning), as well as com- paring the learning curve of a fine-tuned LM to the learning curve of multiple controls, which paints a rich picture of the LM capabil- ities. Our main findings are that: (a) different LMs exhibit qualitatively different reasoning abilities, e.g., ROBERTA succeeds in reason- ing tasks where BERT fails completely; (b) LMs do not reason in an abstract manner and are context-dependent, e.g., while ROBERTA can compare ages, it can do so only when the ages are in the typical range of human ages; (c) On half of our reasoning tasks all models fail completely. Our findings and infrastruc- ture can help future work on designing new datasets, models, and objective functions for pre-training. 1 Introduction Large pre-trained language models (LMs) have revolutionized the field of natural language pro- cessing in the last few years (Dai and Le, 2015; Peters et al., 2018a; Yang et al., 2019; Radford et al., 2019; Devlin et al., 2019). This has insti- gated research exploring what is captured by the contextualized representations that these LMs compute, revealing that they encode substantial amounts of syntax and semantics (Linzen et al., 2016b; Tenney et al., 2019b, a; Shwartz and Dagan, 2019; Lin et al., 2019; Coenen et al., 2019). Despite these efforts, it remains unclear what symbolic reasoning capabilities are difficult to learn from an LM objective only. In this paper, we propose a diverse set of probing tasks for types of symbolic reasoning that are potentially difficult to capture using a LM objective (see Table 1). Our intuition is that because a LM objective focuses on word co-occurrence, it will struggle with tasks that are considered to involve symbolic reasoning such as determining whether a conjunction of properties is held by an object, and comparing the sizes of different objects. Understanding what is missing from current LMs may help design datasets and objectives that will endow models with the missing capabilities. However, how does one verify whether pre- trained representations hold information that is useful for a particular task? Past work mostly resorted to fixing the representations and fine- tuning a simple, often linear, randomly initialized probe, to determine whether the representations hold relevant information (Ettinger et al., 2016; Adi …

Content truncated to 500 words