Mitigating Label Biases for In-context Learning

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 14014–14031 July 9-14, 2023 ©2023 Association for Computational Linguistics Mitigating Label Biases for In-context Learning Yu Fei†1, Yifan Hou∗2, Zeming Chen∗3, Antoine Bosselut3 1UC Irvine, 2ETH Zurich, 3NLP Lab, IC, EPFL, Switzerland yu.fei@uci.edu, yifan.hou@inf.ethz.ch, {zeming.chen, antoine.bosselut}@epfl.ch Abstract Various design settings for in-context learning (ICL), such as the choice and order of the in- context examples, can bias the model’s predic- tions. While many studies discuss these design choices, there have been few systematic inves- tigations into categorizing them and mitigating their impact. In this work, we define a typol- ogy for three types of label biases in ICL for text classification: vanilla-label bias, context- label bias, and domain-label bias (which we conceptualize and detect for the first time). Our analysis demonstrates that prior label bias calibration methods fall short of addressing all three types of biases. Specifically, domain- label bias restricts LLMs to random-level per- formance on many tasks regardless of the choice of in-context examples. To mitigate the effect of these biases, we propose a sim- ple bias calibration method that estimates a language model’s label bias using random in- domain words from the task corpus. After controlling for this estimated bias when mak- ing predictions, our novel domain-context cali- bration significantly improves the ICL perfor- mance of GPT-J and GPT-3 on a wide range of tasks. The gain is substantial on tasks with large domain-label bias (up to 37% in Macro- F1). Furthermore, our results generalize to models with different scales, pretraining meth- ods, and manually-designed task instructions, showing the prevalence of label biases in ICL. 1 Introduction Large language models (LLMs) can perform un- seen tasks by conditioning on a context prompt that consists of a few training example-label pairs (Brown et al., 2020). However, such in-context learning ability is highly sensitive to various design settings, such as the choice (Liu et al., 2021) and order (Lu et al., 2021) of the in-context samples. *Equal contribution. †Work done while interning at EPFL. Figure 1: An illustration of domain-label bias. (a) On Tweet hate, seeing random in-domain (i.d.) words sampled from the dataset severely biases the model towards predicting label hate, while random English (Eng.) words show no such preference. (b) On a movie review dataset, SST-2, we do not observe such bias. Recently, Zhao et al. (2021) showed that the insta- bility of ICL largely arises from the fact that these design settings bias the model toward predicting certain answers (e.g., LLMs often predict the la- bel of the last in-context example). As a result, the sensitivity of the results in ICL studies calls for a systematic discussion of biases in ICL and new methods to properly categorize, detect, and comprehensively mitigate various types of biases. In this work, we conduct a thorough investiga- tion of biases in ICL for text classification. We start by defining a typology of three types of label biases (the model’s undesirable preference toward certain …

Content truncated to 500 words