Privacy Regularization: Joint Privacy-Utility Optimization in LanguageModels

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3799–3807 June 6–11, 2021. ©2021 Association for Computational Linguistics 3799 Privacy Regularization: Joint Privacy-Utility Optimization in Language Models Fatemehsadat Mireshghallah1∗, Huseyin A. Inan3, Marcello Hasegawa2, Victor Rühle2, Taylor Berg-Kirkpatrick1, Robert Sim3 1 University of California San Diego, 2 Microsoft Corporation, 3 Microsoft Research {fatemeh, tberg}@ucsd.edu, {huseyin.inan, marcellh, virueh, rsim}@microsoft.com Abstract Neural language models are known to have a high capacity for memorization of training samples. This may have serious privacy im- plications when training models on user con- tent such as email correspondence. Differen- tial privacy (DP), a popular choice to train models with privacy guarantees, comes with significant costs in terms of utility degrada- tion and disparate impact on subgroups of users. In this work, we introduce two privacy- preserving regularization methods for training language models that enable joint optimiza- tion of utility and privacy through (1) the use of a discriminator and (2) the inclusion of a novel triplet-loss term. We compare our meth- ods with DP through extensive evaluation. We show the advantages of our regularizers with favorable utility-privacy trade-off, faster train- ing with the ability to tap into existing op- timization approaches, and ensuring uniform treatment of under-represented subgroups. 1 Introduction Neural language models (Bengio et al., 2003; Mikolov et al., 2010) have recently seen signif- icant gains in capabilities, and are deployed at scale in several real-world scenarios (Chen et al., 2019; Adam et al., 2020). Training these models on domain-specific user data can further improve their utility. The volume of data required, cou- pled with the inherent sparsity of natural language which often means all data are unique, opens the door to an array of privacy attacks against mod- els and their training data. Sample memorization poses a substantial risk by enabling model inver- sion attacks (Carlini et al., 2020; Ramaswamy et al., 2020; Inan et al., 2021). In these attacks, a curious or malevolent user can query a pre-trained language model on any data record with the intention of re- constructing (parts of) training samples1. ∗ Work done as part of an MSR internship. 1A naïve example is an attacker querying “My account number is” and hoping to receive a user’s account number. Differential Privacy (DP) (Dwork, 2006) is the gold standard approach to address this issue, thanks to its strong and rigorous privacy guarantees. DP- SGD (Abadi et al., 2016) is a popular method to train neural models with differential privacy guar- antees and it works by clipping of the gradients and addition of noise in each update, which provides worst-case guarantees that reflect the likelihood of leaking any attribute of any member of the dataset into the trained model. The worst-case guaran- tees of differential privacy are not customizable, in other words, they cannot be relaxed to protect only certain attributes. Therefore, DP incurs significant loss to model utility (Tramèr and Boneh, 2020). DP training of models is also much slower, …

Content truncated to 500 words