Differential Privacy for Text Analytics via Natural Text Sanitization

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3853–3866 August 1–6, 2021. ©2021 Association for Computational Linguistics 3853 Differential Privacy for Text Analytics via Natural Text Sanitization† Xiang Yue1,‡, Minxin Du2, Tianhao Wang3, Yaliang Li4, Huan Sun1, and Sherman S. M. Chow2 1The Ohio State University 2The Chinese University of Hong Kong 3Carnegie Mellon University 4Alibaba Group {yue.149,sun.397}@osu.edu,{dm018,sherman}@ie.cuhk.edu.hk tianhao@cmu.edu, yaliang.li@alibaba-inc.com Abstract Texts convey sophisticated knowledge. How- ever, texts also convey sensitive information. Despite the success of general-purpose lan- guage models and domain-specific mecha- nisms with differential privacy (DP), existing text sanitization mechanisms still provide low utility, as cursed by the high-dimensional text representation. The companion issue of uti- lizing sanitized texts for downstream analyt- ics is also under-explored. This paper takes a direct approach to text sanitization. Our in- sight is to consider both sensitivity and similar- ity via our new local DP notion. The sanitized texts also contribute to our sanitization-aware pretraining and fine-tuning, enabling privacy- preserving natural language processing over the BERT language model with promising util- ity. Surprisingly, the high utility does not boost up the success rate of inference attacks. 1 Introduction Natural language processing (NLP) requires a lot of training data, which can be sensitive. Na¨ıve redaction approaches (e.g., removing common personally identifiable information) is known to fail (Sweeney, 2015): innocuous-looking fields can be linked to other information sources for reiden- tification. The recent success of many language models (LMs) has motivated security researchers to devise advanced privacy attacks. Carlini et al. (2020b) recover texts from (a single document of) the training data via querying to an LM pretrained from it. Pan et al. (2020) and Song and Raghu- nathan (2020) target the text embedding, e.g., re- vealing from an encoded query to an NLP service. †Our code is available at https://github.com/ xiangyue9607/SanText. ‡The first two authors contributed equally. Raw Private Text Sanitized Private Text Fine-tuning for Downstream Tasks Sanitized Public Corpus Raw Public Corpus "Hannah withdrew $200 from the bank" "He got ¥ 950 from the ATM" Masked Corpus for LM "He got ¥ 950 from the [MASK]" Sanitization-aware Fine-tuning (Service Provider Side) Sanitization (User Side) Sanitization-aware Pretraining (Service Provider Side) Sanitization Mechanism (SanText / SanText+) Language Model became Sanitization Aware! SanText / User SanText+ Masked LM Prediction [MASK] = ["bank"] Sanitized Private Query Classification Regression ... ... Noise Figure 1: Workflow of our PPNLP pipeline, including the user-side sanitization and the service provider-side NLP modeling with pretraining/fine-tuning Emerging NLP works focus on only specific document-level (statistical) features (Weggenmann and Kerschbaum, 2018) or producing private text representations (Xie et al., 2017; Coavoux et al., 2018; Elazar and Goldberg, 2018; Li et al., 2018) as initial solutions to the first issue above on training- data privacy. However, the learned representations are not human-readable, which makes transparency (e.g., required by GDPR) questionable: an average user may not have the technical know-how to verify whether sensitive attributes have been removed or not. Moreover, consider the whole NLP pipeline, the learned representations often entail extra model- …

Content truncated to 500 words