Measuring Inductive Biases of In-Context Learning with Underspecified Demonstrations

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 11289–11310 July 9-14, 2023 ©2023 Association for Computational Linguistics Measuring Inductive Biases of In-Context Learning with Underspecified Demonstrations Chenglei Si1∗ Dan Friedman2∗ Nitish Joshi3 Shi Feng4 Danqi Chen2 He He3 1University of Maryland 2Princeton University 3New York University 4University of Chicago clsi@umd.edu, dfriedman@cs.princeton.edu Abstract In-context learning (ICL) is an important paradigm for adapting large language models (LLMs) to new tasks, but the generalization be- havior of ICL remains poorly understood. We investigate the inductive biases of ICL from the perspective of feature bias: which feature ICL is more likely to use given a set of under- specified demonstrations in which two features are equally predictive of the labels. First, we characterize the feature biases of GPT-3 models by constructing underspecified demonstrations from a range of NLP datasets and feature com- binations. We find that LLMs exhibit clear feature biases—for example, demonstrating a strong bias to predict labels according to senti- ment rather than shallow lexical features, like punctuation. Second, we evaluate the effect of different interventions that are designed to impose an inductive bias in favor of a particu- lar feature, such as adding a natural language instruction or using semantically relevant la- bel words. We find that, while many interven- tions can influence the learner to prefer a par- ticular feature, it can be difficult to overcome strong prior biases. Overall, our results provide a broader picture of the types of features that ICL may be more likely to exploit and how to impose inductive biases that are better aligned with the intended task.1 1 Introduction In-context learning (ICL) is an increasingly popu- lar paradigm for adapting large language models (LLMs) to downstream tasks (Brown et al., 2020). It works by prompting LLMs with a small set of examples that demonstrate the input and output of a task, without requiring any parameter updates. However, a key limitation of ICL is that it can only incorporate a small number of demonstration ex- amples (e.g., 16) due to the context length limit of *Equal contribution. 1Our code and data are publicly available at https://github.com/NoviScl/AmbigPrompt. Sentence Label I've seen this movie more than once and it's worth it. 1 The most disappointing Chinese take-out I've had. 0 Soup was served cold twice even after we complained. 0 Great movie, great actors, great soundtrack! I loved it! 1 GPT-3 Two hypotheses: h1: Sentiment positive - 1, negative - 0 h2: Topic movie - 1, food - 0 Demonstrations Test example The steak is incredible, and is reasonably priced. Words can’t describe how utterly abysmal this movie is. 0 1 Without intervention h1: 92.4% h2: 7.6% With instruction: “Classify based on the topic.” h1: 1.1% h2: 98.9% Better verbalizers: “1” → “movie”, “0” → “food” h1: 0.5% h2: 99.5% Model preference 1 0 h1 h2 Figure 1: We prompt language models with underspeci- fied demonstrations in which two features are equally predictive of the label. In this …

Content truncated to 500 words