On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Study

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 6618–6633 August 1–6, 2021. ©2021 Association for Computational Linguistics 6618 On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Study Divyansh Kaushik†, Douwe Kiela‡, Zachary C. Lipton†, Wen-tau Yih‡ † Carnegie Mellon University; ‡ Facebook AI Research {dkaushik,zlipton}@cmu.edu, {dkiela,scottyih}@fb.com Abstract In adversarial data collection (ADC), a hu- man workforce interacts with a model in real time, attempting to produce examples that elicit incorrect predictions. Researchers hope that models trained on these more challeng- ing datasets will rely less on superficial pat- terns, and thus be less brittle. However, de- spite ADC’s intuitive appeal, it remains un- clear when training on adversarial datasets pro- duces more robust models. In this paper, we conduct a large-scale controlled study focused on question answering, assigning workers at random to compose questions either (i) adver- sarially (with a model in the loop); or (ii) in the standard fashion (without a model). Across a variety of models and datasets, we find that models trained on adversarial data usually per- form better on other adversarial datasets but worse on a diverse collection of out-of-domain evaluation sets. Finally, we provide a qualita- tive analysis of adversarial (vs standard) data, identifying key differences and offering guid- ance for future research.1 1 Introduction Across such diverse natural language processing (NLP) tasks as natural language inference (NLI; Poliak et al., 2018; Gururangan et al., 2018), ques- tion answering (QA; Kaushik and Lipton, 2018), and sentiment analysis (Kaushik et al., 2020), re- searchers have discovered that models can succeed on popular benchmarks by exploiting spurious as- sociations that characterize a particular dataset but do not hold more widely. Despite performing well on independent and identically distributed (i.i.d.) data, these models are liable under plausible do- main shifts. With the goal of providing more chal- lenging benchmarks that require this stronger form of generalization, an emerging line of research has 1Data collected during this study is publicly available at https://github.com/facebookresearch/aqa-study. investigated adversarial data collection (ADC), a scheme in which a worker interacts with a model (in real time), attempting to produce examples that elicit incorrect predictions (e.g., Dua et al., 2019; Nie et al., 2020). The hope is that by identifying parts of the input domain where the model fails one might make the model more robust. Researchers have shown that models trained on ADC perform better on such adversarially collected data and that with successive rounds of ADC, crowdworkers are less able to fool the models (Dinan et al., 2019). While adversarial data may indeed provide more challenging benchmarks, the process and its actual benefits vis-a-vis tasks of interest remain poorly understood, raising several key questions: (i) do the resulting models typically generalize better out of distribution compared to standard data collection (SDC)?; (ii) how much can differences between ADC and SDC be attributed to the way workers behave when attempting to fool …

Content truncated to 500 words