Targeted Data Generation: Finding and Fixing Model Weaknesses

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 8506–8520 July 9-14, 2023 ©2023 Association for Computational Linguistics Targeted Data Generation: Finding and Fixing Model Weaknesses Zexue He∗ UC San Diego La Jolla, CA, USA zehe@eng.ucsd.edu Marco Tulio Ribeiro Microsoft Redmond, WA, USA marcotcr@microsoft.com Fereshte Khani Microsoft Redmond, WA, USA fkhani@microsoft.com Abstract Even when aggregate accuracy is high, state- of-the-art NLP models often fail systematically on specific subgroups of data, resulting in un- fair outcomes and eroding user trust. Addi- tional data collection may not help in address- ing these weaknesses, as such challenging sub- groups may be unknown to users, and under- represented in the existing and new data. We propose Targeted Data Generation (TDG), a framework that automatically identifies chal- lenging subgroups, and generates new data for those subgroups using large language models (LLMs) with a human in the loop. TDG esti- mates the expected benefit and potential harm of data augmentation for each subgroup, and selects the ones most likely to improve within- group performance without hurting overall per- formance. In our experiments, TDG1 signifi- cantly improves the accuracy on challenging subgroups for state-of-the-art sentiment anal- ysis and natural language inference models, while also improving overall test accuracy. 1 Introduction Despite very high accuracy, state-of-the-art NLP models still exhibit systematic failures on specific subgroups of data. For example, Rajani et al. (2022) found that a 95%-accurate sentiment analy- sis model did much worse on club reviews (90%) and movie theater reviews (85%), while Stuart-Ulin (2018) notes how a commercial chatbot avoids any engagement on topics that even mention Islam or the middle east. The existence of these challenging subgroups can lead to unfair outcomes, erode user trust, and ultimately limit deployment of models, even when aggregate accuracy is very high. One possible solution is to collect or generate more data. However, the additional data may still under-sample from specific challenging subgroups, ∗Work done during the internship at Microsoft. 1Codes and collected data will be released in https:// github.com/ZexueHe/TDG. even if data collection is adversarial (Kiela et al., 2021), especially when subgroups are not imme- diately obvious or salient to humans. Therefore it helps little in addressing these weaknesses. Tools for discovering challenging subgroups still require human creativity and effort (Rajani et al., 2022). Khani and Ribeiro (2023); Ribeiro and Lundberg (2022) show that experts are able to improve exist- ing subgroups via careful data augmentation with large language models (LLMs), but finding such challenging subgroups still requires human inge- nuity. Perhaps more importantly, they find that naively augmenting certain subgroups can drasti- cally hurt other subgroups and overall performance (Ribeiro and Lundberg, 2022). Hence, the chal- lenge is not only to find challenging subgroups, but also to determine which subgroups are amenable to data augmentation, and how to augment them effectively. In this work, we propose Targeted Data Gener- ation (TDG), a framework to automatically iden- tify challenging subgroups that can benefit from more data, and then generate that …

Content truncated to 500 words