Knowledge Unlearning for Mitigating Privacy Risks in Language Models

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 14389–14408 July 9-14, 2023 ©2023 Association for Computational Linguistics Knowledge Unlearning for Mitigating Privacy Risks in Language Models Joel Jang1∗ Dongkeun Yoon 1 Sohee Yang1 Sungmin Cha3 Moontae Lee2,4 Lajanugen Logeswaran2 Minjoon Seo1 1KAIST 2LG AI Research 3 Seoul National University 4University of Illinois joeljang@kaist.ac.kr Abstract Pretrained Language Models (LMs) memorize a vast amount of knowledge during initial pre- training, including information that may vio- late the privacy of personal lives and identities. Previous work addressing privacy issues for LMs has mostly focused on data preprocessing and differential privacy methods, both requir- ing re-training the underlying LM. We propose knowledge unlearning as an alternative method to reduce privacy risks for LMs post hoc. We show that simply performing gradient ascent on target token sequences is effective at for- getting them with little to no degradation of general language modeling performances for larger-sized LMs. We also find that sequential unlearning is better than trying to unlearn all the data at once and that unlearning is highly dependent on which kind of data (domain) is forgotten. By showing comparisons with pre- vious methods known to mitigate privacy risks for LMs, we show that our approach can give a stronger empirical privacy guarantee in sce- narios where the data vulnerable to extraction attacks are known a priori while being much more efficient and robust 1. 1 Introduction Recent work has shown that an adversary can ex- tract training data from Pretrained Language Mod- els (LMs) including Personally Identifiable Infor- mation (PII) such as names, phone numbers, and email addresses, and other information such as li- censed code, private clinical notes, and 128-bit UUIDs (Carlini et al., 2021; Lee et al., 2022; Huang et al., 2022; Lehman et al., 2021). In 2021, an AI chatbot Iruda became the first AI system to be sued for violating the Personal Information Protection Act after generating the exact home addresses and bank account numbers of actual individuals unin- tentionally (Park, 2021). Heikkilä (2022) has also ∗work done during internship at LG AI Research. 1We release the code and dataset needed to replicate our results at https://github.com/joeljang/knowledge-unlearning. shown that GPT-3 (Brown et al., 2020), one of the most well-known LM currently in commercial use, offered detailed private information about the Editor-in-Chief of MIT Technology Review includ- ing his family members, work address, and phone number. Considering findings that show extract- ing training data gets easier as LMs scale to larger sizes (Carlini et al., 2022a) and that it is common practice for practitioners to release billion parame- ters pretrained LMs for public use (Gao et al., 2020; Black et al., 2021; Zhang et al., 2022), it has be- come important to provide privacy guarantees for large LMs. Practitioners are required to delete personal in- formation from the LMs by individuals’ request because each individual has the “Right To Be For- gotten (RTBF)" (Mantelero, 2013; Graves et al., 2021) and can limit …

Content truncated to 500 words