Differentially Private Language Models for Secure Data Sharing

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4860 - 4873 December 7-11, 2022 ©2022 Association for Computational Linguistics Differentially Private Language Models for Secure Data Sharing Justus Mattern RWTH Aachen justus.mattern@rwth-aachen.de Zhijing Jin MPI & ETH Zürich zjin@tue.mpg.de Benjamin Weggenmann SAP Security Research benjamin.weggenmann@sap.com Bernhard Schölkopf ∗ MPI for Intelligent Systems bs@tue.mpg.de Mrinmaya Sachan ∗ ETH Zürich msachan@ethz.ch Abstract To protect the privacy of individuals whose data is being shared, it is of high importance to de- velop methods allowing researchers and com- panies to release textual data while providing formal privacy guarantees to its originators. In the field of NLP, substantial efforts have been directed at building mechanisms following the framework of local differential privacy, thereby anonymizing individual text samples before re- leasing them. In practice, these approaches are often dissatisfying in terms of the qual- ity of their output language due to the strong noise required for local differential privacy. In this paper, we approach the problem at hand using global differential privacy, particularly by training a generative language model in a differentially private manner and consequently sampling data from it. Using natural language prompts and a new prompt-mismatch loss, we are able to create highly accurate and fluent textual datasets taking on specific desired at- tributes such as sentiment or topic and resem- bling statistical properties of the training data. We perform thorough experiments indicating that our synthetic datasets do not leak infor- mation from our original data and are of high language quality and highly suitable for train- ing models for further analysis on real-world data. Notably, we also demonstrate that train- ing classifiers on private synthetic data outper- forms directly training classifiers on real data with DP-SGD.1 1 Introduction Rapid advancements in the field of deep learn- ing and natural language processing (NLP) have enabled companies, public institutions and re- searchers to extract information and gain knowl- edge from large-scale data generated by individu- als. In many cases, it is desirable to share such data ∗Equal Supervision. 1Our code is available at https://github.com/ justusmattern/private-datasets-with-llms. GPT-2 Prompt-Based DP Finetuning Sensitive Data Shareable, Anonymous Data Write a [sent.] review about a [prod.]: [Rev.] Review Product Sentiment Cool novel! Book Positive Doesn't charge Electronics Negative Really useful! Electronics Positive Review Product Sentiment Dumb device Electronics Negative What a book! Book Positive Great monitor Electronics Positive GPT-2 Prompt-Based Generation Internal: External: LLM DP LLM Write a [sent.] review about a [prod.]: [Rev.] Figure 1: Main idea of our paper: To share potentially sensitive datasets with third parties, we train a language model (LM) on the sensitive data in a differentially private manner and consequently prompt the LM to generate synthetic samples with privacy guarantees. with third parties, for example when analyses are performed by external consultants or in order to provide high quality benchmarks for the research community. This, however, entails a variety of risks related to privacy that cannot merely be solved by pseudonymization: A variety of deanonymization attacks enable the re-identification …

Content truncated to 500 words