TextHide: Tackling Data Privacy in Language Understanding Tasks

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1368–1382 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 1368 TextHide: Tackling Data Privacy in Language Understanding Tasks Yangsibo Huang† Zhao Song†‡ Danqi Chen† Kai Li† Sanjeev Arora†‡ †Princeton University ‡Institute for Advanced Study {yangsibo, zhaos}@princeton.edu {danqic, li, arora}@cs.princeton.edu Abstract An unsolved challenge in distributed or feder- ated learning is to effectively mitigate privacy risks without slowing down training or reduc- ing accuracy. In this paper, we propose Tex- tHide aiming at addressing this challenge for natural language understanding tasks. It re- quires all participants to add a simple encryp- tion step to prevent an eavesdropping attacker from recovering private text data. Such an en- cryption step is efficient and only affects the task performance slightly. In addition, Tex- tHide fits well with the popular framework of fine-tuning pre-trained language models (e.g., BERT) for any sentence or sentence-pair task. We evaluate TextHide on the GLUE bench- mark, and our experiments show that TextHide can effectively defend attacks on shared gradi- ents or representations and the averaged accu- racy reduction is only 1.9%. We also present an analysis of the security of TextHide using a conjecture about the computational intractabil- ity of a mathematical problem.1 1 Introduction Data privacy for deep learning has become a chal- lenging problem for many application domains in- cluding Natural Language Processing. For ex- ample, healthcare institutions train diagnosis sys- tems on private patients’ data (Pham et al., 2017; Xiao et al., 2018). Google trains a deep learn- ing model for next-word prediction to improve its virtual keyboard using users’ mobile device data (Hard et al., 2018). Such data are decen- tralized but moving them to a centralized location for training a model may violate regulations such as Health Insurance Portability and Accountabil- ity Act (HIPAA) (Act, 1996) and California Con- sumer Privacy Act (CCPA) (Legislature, 2018). 1Our code is available at https://github.com/ Hazelsuko07/TextHide. Federated learning (McMahan et al., year; Kairouz et al., 2019) allows multiple parties train- ing a global neural network model collaboratively in a distributed environment without moving data to a centralized storage. It lets each partici- pant compute a model update (i.e., gradients) on its local data using the latest copy of the global model, and then send the update to the coordinat- ing server. The server then aggregates these up- dates (typically by averaging) to construct an im- proved global model. Privacy has many interpretations depending on the assumed threat models (Kairouz et al., 2019). This paper assumes an eavesdropping attacker with access to all information communicated by all parties, which includes the parameters of the model being trained. With such a threat model, a recent work (Zhu et al., 2019) suggests that an attacker can reverse-engineer the private input. Multi-party computation (Yao, 1982) or homo- morphic encryption (Gentry, 2009) can ensure full privacy but they slow down computations by sev- eral orders of magnitude. Differential privacy (DP) approach (Dwork et al., 2006; Dwork, 2009) is another general framework …

Content truncated to 500 words