Differentially Private Representation for NLP: Formal Guarantee and An Empirical Study on Privacy and Fairness

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2355–2365 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 2355 Differentially Private Representation for NLP: Formal Guarantee and An Empirical Study on Privacy and Fairness Lingjuan Lyu1, Xuanli He† and Yitong Li∗ 1National University of Singapore, lyulj@comp.nus.edu.sg †Monash University, xuanli.he1@monash.edu ∗The University of Melbourne, yitongl4@student.unimelb.edu.au Abstract It has been demonstrated that hidden represen- tation learned by deep model can encode pri- vate information of the input, hence can be ex- ploited to recover such information with rea- sonable accuracy. To address this issue, we propose a novel approach called Differentially Private Neural Representation (DPNR) to pre- serve privacy of the extracted representation from text. DPNR utilises Differential Privacy (DP) to provide formal privacy guarantee. Fur- ther, we show that masking words via dropout can further enhance privacy. To maintain util- ity of the learned representation, we integrate DP-noisy representation into a robust training process to derive a robust target model, which also helps for model fairness over various demographic variables. Experimental results on benchmark datasets under various param- eter settings demonstrate that DPNR largely reduces privacy leakage without significantly sacrificing the main task performance. 1 Introduction Many language applications have involved deep learning techniques to learn text representation through neural models (Bengio et al., 2003; Mikolov et al., 2013; Devlin et al., 2019), perform- ing composition over the learned representation for downstream tasks (Collobert et al., 2011; Socher et al., 2013). However, the input text often pro- vides sufficient clues to portray the author, such as gender, age, and other important attributes. For example, sentiment analysis tasks often have pri- vacy implications for authors whose text is used to train models. Many user attributes have been shown to be easily detectable from online review data, as used extensively in sentiment analysis re- sults (Hovy et al., 2015; Potthast et al., 2017). Pri- vate information can take the form of key phrases explicitly contained in the text. However, it can also be implicit. For example, demographic infor- mation about the author of a text can be predicted with above chance accuracy from linguistic cues in the text itself (Preot¸iuc-Pietro et al., 2015). On the other hand, even the learned representa- tion, rather than the text itself, may still contain sensitive information and incur significant privacy leakage. One might argue that sensitive informa- tion like gender, age, location and password should not be leaked out and should have been removed from representation. However, on the intermedi- ate representation level, which is trained from the input text to contain useful features for the pre- diction task, it can meanwhile encode personal information which might be exploited for adver- sarial usages, especially a modern deep learning model has vastly more capacity than they need to perform well on their tasks. And, it has been justi- fied that an attacker can recover private variables with higher-than-chance accuracy, only using hid- den representation (Li et al., 2018; Coavoux et al., 2018). Therefore, the fact …

Content truncated to 500 words