Beyond Accuracy: Behavioral Testing of NLP Models with CheckList

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4902–4912 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 4902 Beyond Accuracy: Behavioral Testing of NLP Models with CheckList Marco Tulio Ribeiro1 Tongshuang Wu2 Carlos Guestrin2 Sameer Singh3 1Microsoft Research 2University of Washington 3University of California, Irvine marcotcr@gmail.com {wtshuang,guestrin}@cs.uw.edu sameer@uci.edu Abstract Although measuring held-out accuracy has been the primary approach to evaluate general- ization, it often overestimates the performance of NLP models, while alternative approaches for evaluating models either focus on individ- ual tasks or on specific behaviors. Inspired by principles of behavioral testing in software engineering, we introduce CheckList, a task- agnostic methodology for testing NLP mod- els. CheckList includes a matrix of general linguistic capabilities and test types that facil- itate comprehensive test ideation, as well as a software tool to generate a large and diverse number of test cases quickly. We illustrate the utility of CheckList with tests for three tasks, identifying critical failures in both commercial and state-of-art models. In a user study, a team responsible for a commercial sentiment analy- sis model found new and actionable bugs in an extensively tested model. In another user study, NLP practitioners with CheckList cre- ated twice as many tests, and found almost three times as many bugs as users without it. 1 Introduction One of the primary goals of training NLP models is generalization. Since testing “in the wild” is expensive and does not allow for fast iterations, the standard paradigm for evaluation is using train- validation-test splits to estimate the accuracy of the model, including the use of leader boards to track progress on a task (Rajpurkar et al., 2016). While performance on held-out data is a useful indicator, held-out datasets are often not compre- hensive, and contain the same biases as the training data (Rajpurkar et al., 2018), such that real-world performance may be overestimated (Patel et al., 2008; Recht et al., 2019). Further, by summarizing the performance as a single aggregate statistic, it becomes difficult to figure out where the model is failing, and how to fix it (Wu et al., 2019). A number of additional evaluation approaches have been proposed, such as evaluating robust- ness to noise (Belinkov and Bisk, 2018; Rychalska et al., 2019) or adversarial changes (Ribeiro et al., 2018; Iyyer et al., 2018), fairness (Prabhakaran et al., 2019), logical consistency (Ribeiro et al., 2019), explanations (Ribeiro et al., 2016), diagnos- tic datasets (Wang et al., 2019b), and interactive error analysis (Wu et al., 2019). However, these approaches focus either on individual tasks such as Question Answering or Natural Language Infer- ence, or on a few capabilities (e.g. robustness), and thus do not provide comprehensive guidance on how to evaluate models. Software engineering re- search, on the other hand, has proposed a variety of paradigms and tools for testing complex software systems. In particular, “behavioral testing” (also known as black-box testing) is concerned with test- ing different capabilities of a system by validating the input-output behavior, without …

Content truncated to 500 words