Probabilistic Robustness for Data Filtering

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2950–2959 May 2-6, 2023 ©2023 Association for Computational Linguistics Probabilistic Robustness for Data Filtering Yu Yu* and Abdul Rafae Khan* and Shahram Khadivi † and Jia Xu* * School of Engineering and Science, Steven Institute of Technology, NJ 07030, USA † eBay Inc., Aachen 52064, Germany yyu50@stevens.edu, akhan4@stevens.edu, skhadivi@ebay.com, jxu70@stevens.edu Abstract We introduce our probabilistic robustness re- warded data optimization (PRoDO) approach as a framework to enhance the model’s gener- alization power by selecting training data that optimizes our probabilistic robustness metrics. We use proximal policy optimization (PPO) re- inforcement learning to approximately solve the computationally intractable training sub- set selection problem. The PPO’s reward is defined as our (α, ϵ, γ)-Robustness that mea- sures performance consistency over multiple domains by simulating unknown test sets in real-world scenarios using a leaving-one-out strategy. We demonstrate that our PRoDO ef- fectively filters data that lead to significantly higher prediction accuracy and robustness on unknown-domain test sets. Our experiments achieve up to +17.2% increase of accuracy (+25.5% relatively) in sentiment analysis, and - 28.05 decrease of perplexity (-32.1% relatively) in language modeling. In addition, our prob- abilistic (α, ϵ, γ)-Robustness definition serves as an evaluation metric with higher levels of agreement with human annotations than typical performance-based metrics. 1 Introduction Modern machine learning works with massive amounts of data on a range of tasks like language modeling, object detection, and data mining. Us- ing large amounts of training set to build machine learning systems requires extensive computational resources and creates problems like domain shifts and input noise. These unfiltered training data harm model learning robustness (Frénay and Verleysen, 2013) that leads to prediction errors and serious consequences like self driving car fatality and med- ical misdiagnosis (Tian et al., 2018). One problem causing this model instability is that the model learning is opt for the system’s qual- ity, which is typically evaluated by measuring how close this system’s output of a test set is from its human label using metrics such as accuracy, error rate, perplexity, human evaluation score, and so on. However, such a system performance metric highly depends on the test set’s choice and is thus unreli- able. For instance, if our training set is drawn from the news domain, then the performance on a test set from the news domain (in-domain test set) is usu- ally much higher than that from the Twitter domain (out-of-domain test set). As a result, in NLP, while some systems produce human parity results like the use of a pre-trained Transformer (Hendrycks et al., 2020) on in-domain test sets, these systems are easily corrupted by out-of-domain (OOD) samples from the real world. Existing studies on data selection and robust learning demonstrate a need for test domain knowl- edge during training. Some data selection work (Moore and Lewis, 2010; Kirchhoff and Bilmes, 2014; van der Wees et al., 2017; Fan et al., 2017; Qu et al., 2019; Liu et al., …

Content truncated to 500 words