Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8332 - 8347 December 7-11, 2022 ©2022 Association for Computational Linguistics Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks Fatemehsadat Mireshghallah1, Kartik Goyal2, Archit Uniyal3 Taylor Berg-Kirkpatrick1, Reza Shokri4 1 University of California San Diego, 2 Toyota Technological Institute at Chicago (TTIC) 3 University of Virginia, 4 National University of Singapore [fatemeh, tberg]@ucsd.edu, kartikgo@ttic.edu,a.uniyal@virginia.edu,reza@comp.nus.edu.sg Abstract The wide adoption and application of Masked language models (MLMs) on sensitive data (from legal to medical) necessitates a thorough quantitative investigation into their privacy vulnerabilities. Prior attempts at measuring leakage of MLMs via membership inference attacks have been inconclusive, implying potential robustness of MLMs to privacy at- tacks. In this work, we posit that prior attempts were inconclusive because they based their attack solely on the MLM’s model score. We devise a stronger membership inference attack based on likelihood ratio hypothesis testing that involves an additional reference MLM to more accurately quantify the privacy risks of memorization in MLMs. We show that masked language models are indeed susceptible to likelihood ratio membership inference attacks: Our empirical results, on models trained on medical notes, show that our attack improves the AUC of prior membership inference attacks from 0.66 to an alarmingly high 0.90 level. 1 Introduction BERT-based encoders with Masked Language Mod- eling (MLM) Objectives (Devlin et al., 2018; Liu et al., 2019) have become models of choice for use as pre-trained models for various Natural Language Processing (NLP) classification tasks (Wang et al., 2018; Zhang et al., 2019; Rogers et al., 2020) and have been applied to diverse domains such as dis- ease diagnosis, insurance analysis on financial data, sentiment analysis for improved user experience, etc (Yang et al., 2020; Gu et al., 2021; Lee et al., 2020). Given the sensitivity of the data used to train these models, it is crucial to conceive a framework to systematically evaluate the leakage of training data from these models (Shokri, 2022; Carlini et al., 2019; Murakonda and Shokri, 2020; Mireshghallah et al., 2020), and limit the leakage. The conventional way to measure the leakage of training data from machine learning models is by performing mem- bership inference attacks (Shokri et al., 2017; Nasr et al., 2021), in which the attacker tries to determine whether a given sample was part of the training data of the target model or not. These attacks expose the extent of memorization by the model at the level of individual samples. Prior attempts at performing membership inference and reconstruction attacks on masked language models have either been inconclu- sive (Lehman et al., 2021), or have (wrongly) con- cluded that memorization of sensitive data in MLMs is very limited and these models are more private than their generative counterparts (e.g., autoregres- sive language models) (Vakili and Dalianis, 2021; Jagannatha et al., 2021; Nakamura et al., 2020). We hypothesize that prior MLM attacks have been inconclusive because they rely solely on the target model’s (model under attack) loss …

Content truncated to 500 words