Learning to Model and Ignore Dataset Bias with Mixed Capacity Ensembles

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3031–3045 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 3031 Learning to Model and Ignore Dataset Bias with Mixed Capacity Ensembles Christopher Clark∗ Allen Institute for AI chrisc@allenai.org Mark Yatskar† University of Pennsylvania myatskar@seas.upenn.edu Luke Zettlemoyer University of Washington lsz@cs.uw.edu Abstract Many datasets have been shown to contain in- cidental correlations created by idiosyncrasies in the data collection process. For example, sentence entailment datasets can have spuri- ous word-class correlations if nearly all con- tradiction sentences contain the word “not”, and image recognition datasets can have tell- tale object-background correlations if dogs are always indoors. In this paper, we propose a method that can automatically detect and ig- nore these kinds of dataset-specific patterns, which we call dataset biases. Our method trains a lower capacity model in an ensemble with a higher capacity model. During training, the lower capacity model learns to capture rela- tively shallow correlations, which we hypothe- size are likely to reflect dataset bias. This frees the higher capacity model to focus on patterns that should generalize better. We ensure the models learn non-overlapping approaches by introducing a novel method to make them con- ditionally independent. Importantly, our ap- proach does not require the bias to be known in advance. We evaluate performance on syn- thetic datasets, and four datasets built to pe- nalize models that exploit known biases on textual entailment, visual question answering, and image recognition tasks. We show im- provement in all settings, including a 10 point gain on the visual question answering dataset. 1 Introduction Modern machine learning algorithms have been able to achieve impressive results on complex tasks such as language comprehension or image under- standing. However, recent work has cautioned that this success is often partially due to exploiting in- cidental correlations that were introduced during dataset creation, and are not fundamental to the tar- ∗Work completed at the University of Washington †Work completed at the Allen Institute for AI get task. Examples include textual entailment mod- els learning the word “not” always implies contra- diction (Gururangan et al., 2018), visual question answering (VQA) models learning “2” is almost always the answer to “How many” questions (Jabri et al., 2016), and question answering models select- ing entities that occur near question words irrespec- tive of context (Jia and Liang, 2017). We call these kinds of dataset-specific correla- tions dataset bias. Models that exploit dataset bias can perform well on in-domain data, but will be brittle and perform poorly on out-of-domain or ad- versarial examples. Prior work (Clark et al., 2019; He et al., 2019; Wang et al., 2019; Bahng et al., 2020) has shown it is possible to prevent models from adopting biased methods, but require the bias to be known and carefully modeled in advance, e.g., by assuming access to a pre-specified clas- sifier that uses only the bias to make predictions. In this paper, we present a debiasing method that can achieve similar results, but that automatically …

Content truncated to 500 words