Reliability Testing for Natural Language Processing Systems

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 4153–4169 August 1–6, 2021. ©2021 Association for Computational Linguistics 4153 Reliability Testing for Natural Language Processing Systems Samson Tan§♮∗ Shafiq Joty§‡ Kathy Baxter§ Araz Taeihagh♦♣ Gregory A. Bennett§ Min-Yen Kan♮ §Salesforce Research ‡Nanyang Technological University ♮School of Computing, National University of Singapore ♦Lee Kuan Yew School of Public Policy, National University of Singapore ♣Centre for Trusted Internet and Community, National University of Singapore Abstract Questions of fairness, robustness, and trans- parency are paramount to address before de- ploying NLP systems. Central to these con- cerns is the question of reliability: Can NLP systems reliably treat different demographics fairly and function correctly in diverse and noisy environments? To address this, we argue for the need for reliability testing and contextu- alize it among existing work on improving ac- countability. We show how adversarial attacks can be reframed for this goal, via a framework for developing reliability tests. We argue that reliability testing — with an emphasis on inter- disciplinary collaboration — will enable rigor- ous and targeted testing, and aid in the enact- ment and enforcement of industry standards. 1 Introduction Rigorous testing is critical to ensuring a program works as intended (functionality) when used un- der real-world conditions (reliability). Hence, it is troubling that while natural language technologies are becoming increasingly pervasive in our every- day lives, there is little assurance that these NLP systems will not fail catastrophically or amplify dis- crimination against minority demographics when exposed to input from outside the training distribu- tion. Recent examples include GPT-3 (Brown et al., 2020) agreeing with suggested suicide (Rousseau et al., 2020), the mistranslation of an innocuous social media post resulting in a minority’s arrest (Hern, 2017), and biased grading algorithms that can negatively impact a minority student’s future (Feathers, 2019). Additionally, a lack of rigorous testing, coupled with machine learning’s (ML) im- plicit assumption of identical training and testing distributions, may inadvertently result in systems that discriminate against minorities, who are often underrepresented in the training data. This can take ∗Correspondence to: samson.tan@salesforce.com Figure 1: How DOCTOR can integrate with existing system development workflows. Test (left) and sys- tem development (right) take place in parallel, separate teams. Reliability tests can thus be constructed inde- pendent of the system development team, either by an internal “red team” or by independent auditors. the form of misrepresentation of or poorer perfor- mance for people with disabilities, specific gender, ethnic, age, or linguistic groups (Hovy and Spruit, 2016; Crawford, 2017; Hutchinson et al., 2020). Amongst claims of NLP systems achieving human parity in challenging tasks such as question answer- ing (Yu et al., 2018), machine translation (Has- san et al., 2018), and commonsense inference (De- vlin et al., 2019), research has demonstrated these systems’ fragility to natural and adversarial noise (Goodfellow et al., 2015; Belinkov and Bisk, 2018) and out-of-distribution data (Fisch et al., 2019). It is also still common …

Content truncated to 500 words