Sentence-level Privacy for Document Embeddings

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 3367 - 3380 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Sentence-level Privacy for Document Embeddings Casey Meehan and Khalil Mrini and Kamalika Chaudhuri UC San Diego {cmeehan, kmrini, kamalika}@eng.ucsd.edu Abstract User language data can contain highly sensi- tive personal content. As such, it is impera- tive to offer users a strong and interpretable privacy guarantee when learning from their data. In this work, we propose SentDP: pure local differential privacy at the sentence level for a single user document. We propose a novel technique, DeepCandidate, that com- bines concepts from robust statistics and lan- guage modeling to produce high-dimensional, general-purpose ϵ-SentDP document embed- dings. This guarantees that any single sen- tence in a document can be substituted with any other sentence while keeping the embed- ding ϵ-indistinguishable. Our experiments in- dicate that these private document embeddings are useful for downstream tasks like sentiment analysis and topic classification and even out- perform baseline methods with weaker guaran- tees like word-level Metric DP. 1 Introduction Language models have now become ubiquitous in NLP (Devlin et al., 2019; Liu et al., 2019b; Alsentzer et al., 2019), pushing the state of the art in a variety of tasks (Strubell et al., 2018; Liu et al., 2019a; Mrini et al., 2021). While language models capture meaning and various linguistic properties of text (Jawahar et al., 2019; Yenicelik et al., 2020), an individual’s written text can include highly sen- sitive information. Even if such details are not needed or used, sensitive information has been found to be vulnerable and detectable to attacks (Pan et al., 2020; Abdalla et al., 2020; Carlini et al., 2020). Reconstruction attacks (Xie and Hong, 2021) have even successfully broken through pri- vate learning schemes that rely on encryption-type methods (Huang et al., 2020). As of now, there is no broad agreement on what constitutes good privacy for natural language (Kairouz et al., 2019). Huang et al. (2020) ar- gue that different applications and models require Figure 1: x and x′ yield z ∈ Rd with similar probability. different privacy definitions. Several emerging works propose to apply Metric Differential Privacy (Alvim et al., 2018) at the word level (Feyisetan et al., 2019; Feyisetan and Kasiviswanathan, 2021; Carvalho et al., 2021; Qu et al., 2021; Yue et al., 2021; Xu et al., 2021) . They propose to add noise to word embeddings, such that they are indistin- guishable from their nearest neighbours. At the document level, however, the above defi- nition has two areas for improvement. First, it may not offer the level of privacy desired. Having each word indistinguishable with similar words may not hide higher level concepts in the document, and may not be satisfactory for many users. Second, it may not be very interpretable or easy to com- municate to end-users, since the privacy definition relies fundamentally on the choice of embedding model to determine which words are indistinguish- able with a …

Content truncated to 500 words