TextFusion: Privacy-Preserving Pre-trained Model Inference via Token Fusion

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8360 - 8371 December 7-11, 2022 ©2022 Association for Computational Linguistics TextFusion: Privacy-Preserving Pre-trained Model Inference via Token Fusion Xin Zhou1∗, Jinzhu Lu1∗, Tao Gui2†, Ruotian Ma1, Zichu Fei1 Yuran Wang4, Yong Ding4, Yibo Zhang4, Qi Zhang1†, Xuanjing Huang1, 3 1School of Computer Science, Fudan University, Shanghai, China 2Institute of Modern Languages and Linguistics, Fudan University, Shanghai, China 3 International Human Phenome Institutes, Shanghai, China 4 Honor Device Co., Ltd {xzhou20,tgui,qz,xjhuang}@fudan.edu.cn, lujz21@m.fudan.edu.cn Abstract Recently, more and more pre-trained language models are released as a cloud service. allows users who lack computing resourcesIt to perform inference with a powerful model by uploading data to the cloud. The plain text may contain private information, as the result, users prefer to do partial computations locally and upload intermediate representa- tions to the cloud for subsequent inference. However, recent studies have shown that plain text can also be recovered by intermediate representations with reasonable accuracy, thus the risk of privacy leakage still exists. To address this issue, we propose TextFusion, a novel method for preserving inference privacy. Specifically, we train a Fusion Predictor to dynamically fuse token representations, which hides multiple private token representations behind an unrecognizable one. Furthermore, a misleading training scheme is employed to privatize these representations. In this way, the cloud only receives incomplete and perturbed representations, making it diffi- cult to accurately recover the complete plain text. The experimental results on diverse classification tasks show that our approach can effectively preserve inference privacy without substantially sacrificing performance in different scenarios. 1 Introduction Pre-trained language models (PLMs) achieve state- of-the-art performance in many NLP tasks (Devlin et al., 2018; Liu et al., 2019; Brown et al., 2020; Qiu et al., 2020) . In industrial applications, running full PLM locally can be very expensive or even infeasible for most users due to the high computation requirements. Therefore, PLMs are usually released as cloud services, allowing users to access these powerful models by uploading the data to the cloud (DALE, 2015; Pais et al., 2022). ∗Equal contribution. †Corresponding authors. Users Privacy Attacker Reconstruction Model John lived in 3rd street Cloud Server Prediction Attacker Prediction PLM Inference Representation John lives in 3rd street Local Encoder Local Representation Private Text Figure 1: Illustration of privacy attack when using cloud services. Users upload the local representations to the cloud server for subsequent inference. Attacks can recover these representations to plain text, thus steal private user information. Though inference with cloud models is con- venient and powerful, it brings risk to privacy due to the sensitive nature of user data (Chi et al., 2018). For example, plain text data may contain private information about the user, such as name, address, and phone number. is unacceptable for most users to upload suchIt private data directly to the cloud (Jegorova et al., 2021). A natural way to avoid this problem is to perform affordable computation locally to obtain intermediate representations and upload them to the cloud …

Content truncated to 500 words