CRYPTOGRU: Low Latency Privacy-Preserving Text Analysis With GRU

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2052–2057 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 2052 CRYPTOGRU: Low Latency Privacy-Preserving Text Model Inference Bo Feng and Qian Lou and Lei Jiang and Geoffrey Fox Intelligent Systems Engineering Luddy School of Informatics, Computing, and Engineering Indiana University Bloomington {fengbo,louqian,jiang60}@iu.edu, gcf@indiana.edu Abstract Homomorphic encryption (HE) and garbled circuit (GC) provide the protection for users’ privacy. However, simply mixing the HE and GC in RNN models suffer from long inference latency due to slow activation functions. In this paper, we present a novel hybrid structure of HE and GC gated recurrent unit (GRU) net- work, CRYPTOGRU, for low-latency secure inferences. CRYPTOGRU replaces computa- tionally expensive GC-based tanh with fast GC-based ReLU, and then quantizes sigmoid and ReLU to smaller bit-length to acceler- ate activations in a GRU. We evaluate CRYP- TOGRU with multiple GRU models trained on 4 public datasets. Experimental results show CRYPTOGRU achieves top-notch accu- racy and improves the secure inference latency by up to 138× over one of the state-of-the-art secure networks on the Penn Treebank dataset. 1 Introduction Billions of text analysis requests are processed by powerful RNN models (Bahdanau et al., 2015; Kan- nan et al., 2016) deployed on public clouds every- day. These text analysis requests contain private emails, personal text messages, and sensitive online reviews. For instance, Gmail smart reply genera- tion needs to scan users’ plaintext email messages anonymously (Kannan et al., 2016). Prior work (Juvekar et al., 2018) proposes a hybrid cryptographic scheme that uses homomor- phic encryption (HE) to process linear layers and garbled circuits (GC) to compute activations in a convolutional neural network. Compared to convolutional neural networks (CNN), RNNs can achieve more competitive accuracy in text analysis tasks (Bahdanau et al., 2015; Podschwadt and Tak- abi, 2020; Bakshi and Last, 2020). Mixing HE and GC presents impressing results in secure classifica- tion tasks (Barni et al., 2011). However, mixing HE and GC in RNN will suffer from a long inference latency due to the slow GC-based activations. In contrast to a CNN, a RNN (Bahdanau et al., 2015) requires more types of activations such as tanh and sigmoid. The GC protocol (Ohrimenko et al., 2016) has to use a huge garbled table to implement a tanh or sigmoid activation. Both garbling and evaluating such a large table add significant latency to RNN layers. Based on our experimental and theoretical analysis, the GC-based activations can occupy up to 91% of the inference latency in a HE and GC hybrid secure GRU. To reduce the GC-based activation latency, we propose a novel secure gated recurrent unit (GRU) network framework, CRYPTOGRU, that achieves high security level and low inference latency si- multaneously. We use SIMD HE kerel functions from Juvekar et al. (2018) to process linear opera- tions in a GRU cell, while it adopts GC to compute activations. Our contributions are summarized as follows: • We build a HE and GC hybrid privacy-preserving cryptosystem, CRYPTOGRU, that …

Content truncated to 500 words