ER-AE: Differentially Private Text Generation for Authorship Anonymization

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3997–4007 June 6–11, 2021. ©2021 Association for Computational Linguistics 3997 ER-AE: Differentially Private Text Generation for Authorship Anonymization Haohan Bo McGill University, Canada haohan.bo@mail.mcgill.ca Steven H. H. Ding Queen’s University, Canada ding@cs.queensu.ca Benjamin C. M. Fung McGill University, Canada ben.fung@mcgill.ca Farkhund Iqbal Zayed University, UAE farkhund.iqbal@zu.ac.ae Abstract Most of privacy protection studies for tex- tual data focus on removing explicit sen- sitive identifiers. However, personal writing style, as a strong indicator of the author- ship, is often neglected. Recent studies, such as SynTF, have shown promising results on privacy-preserving text mining. However, their anonymization algorithm can only output nu- meric term vectors which are difficult for the recipients to interpret. We propose a novel text generation model with a two-set exponential mechanism for authorship anonymization. By augmenting the semantic information through a REINFORCE training reward function, the model can generate differentially private text that has a close semantic and similar gram- matical structure to the original text while re- moving personal traits of the writing style. It does not assume any conditioned labels or par- alleled text data for training. We evaluate the performance of the proposed model on the real- life peer reviews dataset and the Yelp review dataset. The result suggests that our model outperforms the state-of-the-art on semantic preservation, authorship obfuscation, and sty- lometric transformation. 1 Introduction Privacy has become a vital issue in online data gathering and public data release. Various machine learning models and privacy preservation algo- rithms have been studied for relational data (John- son et al., 2018), network graph data (Chen et al., 2014), and transactional data (Li et al., 2012). Some of them have been successfully adopted in real-life applications such as telemetry collection (Cortés et al., 2016). However, the studies on privacy pro- tection for textual data are still preliminary. Most related works only focus on replacing the sen- sitive key phrases in the text (Vasudevan and John, 2014) without considering the author’s writ- ing style, which is indeed a strong indicator of a person’s identity. Even though some textual data, such as double-blind academic reviews, is released anonymously, the adversaries may recover the au- thor’s identity using the personal traits in writing. Stylometric techniques (Koppel et al., 2011) can identify an author of the text from 10,000 can- didates. They are effective across online posts, articles, emails, and reviews (Ding et al., 2015, 2017). Nevertheless, traditional text sanitization methods (Narayanan and Shmatikov, 2008) focus on anonymizing the contents, such as patient infor- mation, instead of the writing style, so they are inef- fective against writing style analysis. The original author can be easily re-identified even if protected by these traditional approaches (Iqbal et al., 2008, 2010, 2013; Schmid et al., 2015). Only a few recent studies focus on author- ship anonymization, aiming to hide the personal traits of writing style in the given textual data. Anonymouth (McDonald et al., 2012) …

Content truncated to 500 words