Reconstruction Attack on Instance Encoding for Language Understanding

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2038–2044 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 2038 Reconstruction Attack on Instance Encoding for Language Understanding Shangyu Xie Illinois Institute of Technology sxie14@hawk.iit.edu Yuan Hong Illinois Institute of Technology yuan.hong@iit.edu Abstract A private learning scheme TextHide was re- cently proposed to protect the private text data during the training phase via so-called instance encoding. We propose a novel reconstruction attack to break TextHide by recovering the pri- vate training data, and thus unveil the privacy risks of instance encoding. We have experi- mentally validated the effectiveness of the re- construction attack with two commonly-used datasets for sentence classification. Our attack would advance the development of privacy pre- serving machine learning in the context of nat- ural language processing. 1 Introduction With the development of deep learning technolo- gies, a large number of applications in various do- mains (e.g., image classification and NLP) have been greatly promoted with significantly improved performance. However, this also arouses serious privacy concerns since a large portion of the train- ing data are usually collected from individuals. For instance, the diagnosis systems in hospitals or healthcare institutions will be trained on the pa- tients’ private data, such as medical history (Pham et al., 2017), and radiology medical images (Hosny et al., 2018). In addition, it has been reported that the input keyboard prediction model can be trained with the users’ data on mobile devices (Hard et al., 2018), and the assisted composing function for emails/texts can be trained with users’ personal messages (Chen et al., 2019). There have been various works on protecting users’ data privacy during the training, which are categorized into two main types: 1) composing cryptographic protocols for securely training the data (Bonawitz et al., 2016; Mohassel and Zhang, 2017; Mohassel and Rindal, 2018) which result in high computational and communication costs in general; 2) leveraging the differential privacy tech- niques (Dwork et al., 2006; Chaudhuri et al., 2011; Hong et al., 2015; Abadi et al., 2016) to prevent the information leakage which typically cause sig- nificant accuracy loss. Despite the above demerits, both types of methods can ensure provable privacy guarantees for the training data. This also raises the question: are there any private learning schemes which can preserve both accuracy and efficiency? To this end, there are several techniques (Huang et al., 2020a,b) which privately train the model via the so-called instance encoding scheme, by en- coding the local data into a somewhat “encrypted” (encoded) data with a mixup scheme (Zhang et al., 2018), and directly training the model on the en- coded data. Data privacy is claimed to be well preserved through the encoding method while only causing minor accuracy loss with the merit of the mixup scheme. In this paper, we study the pri- vacy risks of the instance encoding scheme, and show that the instance encoding cannot provide sufficient privacy protection as the conventional cryptographic techniques against well-designed at- tacks. Specifically, we design …

Content truncated to 500 words