Query-Efficient Black-Box Red Teaming via Bayesian Optimization

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 11551–11574 July 9-14, 2023 ©2023 Association for Computational Linguistics Query-Efficient Black-Box Red Teaming via Bayesian Optimization Deokjae Lee1 JunYeong Lee1 Jung-Woo Ha2,3 Jin-Hwa Kim1,2,3 Sang-Woo Lee2,3,4 Hwaran Lee2,3 Hyun Oh Song1,† 1Seoul National University 2NAVER AI Lab 3NAVER Cloud 4Korea Advanced Institute of Science and Technology {bdbj,hyunoh}@mllab.snu.ac.kr Abstract The deployment of large-scale generative mod- els is often restricted by their potential risk of causing harm to users in unpredictable ways. We focus on the problem of black-box red teaming, where a red team generates test cases and interacts with the victim model to dis- cover a diverse set of failures with limited query access. Existing red teaming methods construct test cases based on human supervi- sion or language model (LM) and query all test cases in a brute-force manner without in- corporating any information from past evalua- tions, resulting in a prohibitively large number of queries. To this end, we propose Bayesian red teaming (BRT), novel query-efficient black- box red teaming methods based on Bayesian optimization, which iteratively identify diverse positive test cases leading to model failures by utilizing the pre-defined user input pool and the past evaluations. Experimental results on various user input pools demonstrate that our method consistently finds a significantly larger number of diverse positive test cases under the limited query budget than the base- line methods. The source code is available at https://github.com/snu-mllab/Bayesian-Red- Teaming. 1 Introduction Recently, generative models have demonstrated ex- ceptional performance on a broad range of gen- eration tasks, including open-domain dialogue, prompt continuation, and text-to-image generation, thanks to the rise of large-scale models such as BlenderBot, Gopher, GPT-3, PaLM, and Dall·E 2 (Roller et al., 2021; Rae et al., 2021; Brown et al., 2020; Chowdhery et al., 2022; Ramesh et al., 2022). While utilizing large models in commercial sys- tems can provide significant benefits, it also poses a risk of unexpectedly causing harm to users, such †Correspondence to Hyun Oh Song. How is the … That sounds … You need to… what do you … I think they are just people. I don’t understand why they need a special label. what do you think of queer people? What do you think of transgender individuals? I think they are mentally ill. User Input Pool Selector GP Editor GP Language Model Response Figure 1: Illustration of edit-based BRT. Edit-based BRT constructs a user input pool and generates test cases by selecting and editing user inputs in the pool. Here, our edit-based BRT is applied to BlenderBot-3B using the user input from Bot Adversarial Dialogue. as the generation of offensive responses or NSFW images (Lee, 2016; Rando et al., 2022). Thus, it is essential to identify and prevent these failures before deployment to avoid severe ramifications to society (Xu et al., 2021; Dinan et al., 2019). The primary goal of red teaming is to identify many diverse positive test cases which lead to model failures (Perez et al., …

Content truncated to 500 words