Inoculation by Fine-Tuning: A Method for Analyzing Challenge Datasets

Proceedings of NAACL-HLT 2019, pages 2171–2179 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2171 Inoculation by Fine-Tuning: A Method for Analyzing Challenge Datasets Nelson F. Liu♠♥♣ Roy Schwartz♠♣ Noah A. Smith♠♣ ♠Paul G. Allen School of Computer Science & Engineering, University of Washington, Seattle, WA, USA ♥Department of Linguistics, University of Washington, Seattle, WA, USA ♣Allen Institute for Artificial Intelligence, Seattle, WA, USA {nfliu,roysch,nasmith}@cs.washington.edu Abstract Several datasets have recently been con- structed to expose brittleness in models trained on existing benchmarks. While model perfor- mance on these challenge datasets is signifi- cantly lower compared to the original bench- mark, it is unclear what particular weaknesses they reveal. For example, a challenge dataset may be difficult because it targets phenomena that current models cannot capture, or because it simply exploits blind spots in a model’s spe- cific training set. We introduce inoculation by fine-tuning, a new analysis method for study- ing challenge datasets by exposing models (the metaphorical patient) to a small amount of data from the challenge dataset (a metaphor- ical pathogen) and assessing how well they can adapt. We apply our method to analyze the NLI “stress tests” (Naik et al., 2018) and the Adversarial SQuAD dataset (Jia and Liang, 2017). We show that after slight exposure, some of these datasets are no longer challeng- ing, while others remain difficult. Our results indicate that failures on challenge datasets may lead to very different conclusions about models, training datasets, and the challenge datasets themselves. 1 Introduction NLP research progresses through the construction of dataset-benchmarks and the development of systems whose performance on them can be fairly compared. A recent pattern involves challenges to benchmarks:1 manipulations to input data that re- sult in severe degradation of system performance, but not human performance. These challenges have been used as evidence that current systems are brittle (Belinkov and Bisk, 2018; Mudrakarta et al., 2018; Zhao et al., 2018; Glockner et al., 2018; Ebrahimi et al., 2018; Ribeiro et al., 2018, 1Often referred to as “adversarial datasets” or “attacks”. Figure 1: An illustration of the standard challenge eval- uation procedure (e.g., Jia and Liang, 2017) and our proposed analysis method. “Original” refers to the a standard dataset (e.g., SQuAD) and “Challenge” refers to the challenge dataset (e.g., Adversarial SQuAD). Outcomes are discussed in Section 2. inter alia). For instance, Naik et al. (2018) gen- erated natural language inference challenge data by applying simple textual transformations to ex- isting examples from MultiNLI (Williams et al., 2018) and SNLI (Bowman et al., 2015). Similarly, Jia and Liang (2017) built an adversarial evalua- tion dataset for reading comprehension based on SQuAD (Rajpurkar et al., 2016). What should we conclude when a system fails on a challenge dataset? In some cases, a challenge might exploit blind spots in the design of the origi- nal dataset (dataset weakness). In others, the chal- lenge might expose an inherent inability of a par- ticular model family to handle certain natural lan- guage phenomena (model weakness). …

Content truncated to 500 words