Measuring Robustness for NLP

Proceedings of the 29th International Conference on Computational Linguistics, pages 3908–3916 October 12–17, 2022. 3908 Measuring Robustness for NLP Yu Yu* and Abdul Rafae Khan * and Jia Xu* * School of Engineering and Science, Steven Institute of Technology, NJ 07030, USA yyu50@stevens.edu, akhan4@stevens.edu, jxu70@stevens.edu Abstract The quality of Natural Language Processing (NLP) models is typically measured by the ac- curacy or error rate of a predefined test set. Be- cause the evaluation and optimization of these measures are narrowed down to a specific do- main like news and cannot be generalized to other domains like Twitter, we often observe that a system reported with human parity results generates surprising errors in real-life use sce- narios. We address this weakness with a new approach that uses an NLP quality measure based on robustness. Unlike previous work that has defined robustness using Minimax to bound worst cases, we measure robustness based on the consistency of cross-domain accuracy and introduce the coefficient of variation and (ϵ, γ)- Robustness. Our measures demonstrate higher agreements with human evaluation than accu- racy scores like BLEU on ranking Machine Translation (MT) systems. 1 Introduction Evaluation criteria serve as learning objectives and model assessment standards and are crucial for NLP research. Conventional evaluation methods compute the accuracy or errors compared to the reference on a predefined test set, such as BLEU or TER. Since the evaluation results highly depend on the test set, they may not apply to real-world test in- puts that come from an unknown distribution. For example, one may query a legal document while the system is trained on the news. Therefore, it is necessary to define a measure that can give an idea of how robust system performance will be on unseen test data so that we can predict whether our model is generalizable to new domains. There has been influential investigation into defining robustness instead of accuracy alone (Bas- tani et al., 2016; Hein and Andriushchenko, 2017; Weng et al., 2018). Robustness of a machine learn- ing model can be described as the characteristic of how accurate the model is in making its predic- tions when tested on a new (but similar) dataset. For example, one definition of robustness claims “If a testing sample is similar to a training sam- ple, then the testing error is close to the training error” (Xu and Mannor, 2012). Such testing sam- ples, which are samples dissimilar from training samples are known as adversarial examples. Many studies on robustness measures (Heinze-Deml and Meinshausen, 2017; Araujo et al., 2019; Carlini and Wagner, 2017) focus on the worst-case scenarios with adversarial inputs. However, because the worst-case appears very infrequently, in particular, if artificially simu- lated (Wang et al., 2020; Jin et al., 2020; Alzantot et al., 2018), a worst-case analysis has the inher- ent problem that if the worst case is far from the typical, then the quantification assigns a numeri- cal value to cases that occur rarely, like outliers. For instance, if a system accuracy has a …

Content truncated to 500 words