research-article

HT2ML: : An efficient hybrid framework for privacy-preserving Machine Learning using HE and TEE

Authors:

Giovanni RusselloAuthors Info & Claims

Volume 135, Issue C

https://doi.org/10.1016/j.cose.2023.103509

Published: 10 January 2024 Publication History

Abstract

Outsourcing Machine Learning (ML) tasks to cloud servers is a cost-effective solution when dealing with distributed data. However, outsourcing these tasks to cloud servers could lead to data breaches. Secure computing methods, such as Homomorphic Encryption (HE) and Trusted Execution Environments (TEE), have been used to protect outsourced data. Nevertheless, HE remains inefficient in processing complicated functions (e.g., non-linear functions) and TEE (e.g., Intel SGX) is not ideal for directly processing ML tasks due to side-channel attacks and parallel-unfriendly computation.

In this paper, we propose a hybrid framework integrating SGX and HE, called HT2ML, to protect user's data and models. In HT2ML, HE-friendly functions are protected with HE and performed outside the enclave, while the remaining operations are performed inside the enclave obliviously. HT2ML leverages optimised HE matrix multiplications to accelerate HE computations outside the enclave while using oblivious blocks inside the enclave to prevent access-pattern-based attacks. We evaluate HT2ML using Linear Regression (LR) training and Convolutional Neural Network (CNN) inference as two instantiations. The performance results show that HT2ML is up to ∼11× faster than HE only baseline with 6-dimensional data in LR training. For CNN inference, HT2ML is ∼196× faster than the most recent approach (Xiao et al., ICDCS'21).

References

[1]

A. Ahmad, B. Joe, Y. Xiao, Y. Zhang, I. Shin, B. Lee, OBFUSCURO: a commodity obfuscation engine on intel SGX, in: 26th Annual Network and Distributed System Security Symposium, NDSS 2019, San Diego, California, USA, February, 2019, pp. 24–27. OBFUSCURO: a commodity obfuscation engine on intel SGX, The Internet Society, 2019.

[2]

M.R. Albrecht, R. Player, S. Scott, On the concrete hardness of learning with errors, J. Math. Cryptol. 9 (2015) 169–203.

[3]

Alibaba (2020): Alibaba cloud security white paper. https://www.alibabacloud.com/.

[4]

K.E. Batcher, Sorting networks and their applications, in: Proceedings of the April 30–May 2, 1968, Spring Joint Computer Conference, 1968, pp. 307–314.

[5]

D. Boneh, E.J. Goh, K. Nissim, Evaluating 2-dnf formulas on ciphertexts, in: Theory of Cryptography Conference, Springer, 2005, pp. 325–341.

[6]

R. Bost, R.A. Popa, S. Tu, S. Goldwasser, Machine learning classification over encrypted data, in: NDSS, 2015, p. 4325.

[7]

F. Bourse, M. Minelli, M. Minihold, P. Paillier, Fast homomorphic evaluation of deep discretized neural networks, in: Annual International Cryptology Conference, Springer, 2018, pp. 483–512.

[8]

Z. Brakerski, C. Gentry, V. Vaikuntanathan, (Leveled) fully homomorphic encryption without bootstrapping, ACM Trans. Comput. Theory (TOCT) 6 (2014) 1–36.

[9]

Z. Brakerski, V. Vaikuntanathan, Fully homomorphic encryption from ring-lwe and security for key dependent messages, in: Annual Cryptology Conference, Springer, 2011, pp. 505–524.

[10]

Chabanne, H., De Wargny, A., Milgram, J., Morel, C., Prouff, E., 2017. Privacy-preserving classification on deep neural network. Cryptology ePrint Archive.

[11]

G. Chen, S. Chen, Y. Xiao, Y. Zhang, Z. Lin, T.H. Lai, Sgxpectre: stealing intel secrets from sgx enclaves via speculative execution, in: 2019 IEEE European Symposium on Security and Privacy (EuroS&P), IEEE, 2019, pp. 142–157.

[12]

Cheng, P.C.; Eykholt, K.; Gu, Z.; Jamjoom, H.; Jayaram, K.; Valdez, E.; Verma, A. (2021): Separation of powers in federated learning. ArXiv preprint arXiv:2105.09400.

[13]

J.H. Cheon, A. Kim, M. Kim, Y. Song, Homomorphic encryption for arithmetic of approximate numbers, in: International Conference on the Theory and Application of Cryptology and Information Security, Springer, 2017, pp. 409–437.

[14]

Chollet, F.; et al. (2015): Keras. https://keras.io.

[15]

L. Coppolino, S. D'Antonio, V. Formicola, G. Mazzeo, L. Romano, Vise: combining intel sgx and homomorphic encryption for cloud industrial control systems, IEEE Trans. Comput. 70 (2020) 711–724.

[16]

Corporation, I., 2016. Intel (r) 64 and ia-32 architectures software developer's manual. Combined Volumes, Dec.

[17]

Costan, V., Devadas, S., 2016. Intel sgx explained. IACR Cryptol. ePrint Arch. 2016. pp. 1–118.

[18]

W. Dai, B. Sunar, Cuhe: a homomorphic encryption accelerator library, in: International Conference on Cryptography and Information Security in the Balkans, Springer, 2015, pp. 169–186.

[19]

F. Desai, D. Chowdhury, R. Kaur, M. Peeters, R.C. Arya, G.S. Wander, S.S. Gill, R. Buyya, Healthcloud: a system for monitoring health status of heart patients using machine learning and cloud computing, Int. Things 17 (2022).

[20]

T. ElGamal, A public key cryptosystem and a signature scheme based on discrete logarithms, IEEE Trans. Inf. Theory 31 (1985) 469–472.

[21]

S. Fan, Z. Wang, W. Xu, R. Hou, D. Meng, M. Zhang, Tensorfhe: achieving practical computation on encrypted data using gpgpu, in: 2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA), IEEE, 2023, pp. 922–934.

[22]

C. Gentry, et al., A Fully Homomorphic Encryption Scheme, vol. 20, Stanford University, Stanford, 2009.

[23]

R. Gilad-Bachrach, N. Dowlin, K. Laine, K. Lauter, M. Naehrig, J. Wernsing, Cryptonets: applying neural networks to encrypted data with high throughput and accuracy, in: International Conference on Machine Learning, PMLR, 2016, pp. 201–210.

[24]

G.H. Golub, C.F. Van Loan, Matrix Computations, JHU Press, Baltimore, 2013.

[25]

Google (2022): Google prediction API. https://cloud.google.com/vertex-ai.

[26]

T. Graepel, K. Lauter, M. Naehrig, Ml confidential: machine learning on encrypted data, in: International Conference on Information Security and Cryptology, Springer, 2012, pp. 1–21.

[27]

Granlund, T., 1996. Gnu mp. The GNU Multiple Precision Arithmetic Library 2.

[28]

D. Gruss, M. Lipp, M. Schwarz, D. Genkin, J. Juffinger, S. O'Connell, W. Schoechl, Y. Yarom, Another flip in the wall of rowhammer defenses, in: 2018 IEEE Symposium on Security and Privacy (SP), IEEE, 2018, pp. 245–261.

[29]

Halevi, S., Shoup, V., 2020. Design and implementation of helib: a homomorphic encryption library. IACR Cryptol. ePrint Arch. 2020. p. 1481.

[30]

Huang, Z., Lu, W.j., Hong C., Ding, J., 2022. Cheetah: Lean and fast secure two-party deep neural network inference. Cryptology ePrint Archive.

[31]

Y. Jia, Learning Semantic Image Representations at a Large Scale, University of California, Berkeley, 2014.

[32]

X. Jiang, M. Kim, K. Lauter, Y. Song, Secure outsourced matrix computation and application to neural networks, in: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, 2018, pp. 1209–1222.

[33]

C. Juvekar, V. Vaikuntanathan, A. Chandrakasan, {GAZELLE}: a low latency framework for secure neural network inference, in: 27th {USENIX} Security Symposium ({USENIX} Security 18), 2018, pp. 1651–1669.

[34]

A. Law, C. Leung, R. Poddar, R.A. Popa, C. Shi, O. Sima, C. Yu, X. Zhang, W. Zheng, Secure collaborative training and inference for xgboost, in: Proceedings of the 2020 Workshop on Privacy-Preserving Machine Learning in Practice, 2020, pp. 21–26.

[35]

LeCun, Y. (1998): The mnist database of handwritten digits. http://yann.lecun.com/exdb/mnist/.

[36]

Microsoft (2022): Microsoft azure machine learning. https://azure.microsoft.com/en-us/services/machine-learning/.

[37]

Microsoft (2022): Open enclave SDK. https://openenclave.io.

[38]

P. Mohassel, P. Rindal, Aby3: a mixed protocol framework for machine learning, in: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, 2018, pp. 35–52.

[39]

P. Mohassel, Y. Zhang, Secureml: a system for scalable privacy-preserving machine learning, in: 2017 IEEE Symposium on Security and Privacy (SP), IEEE, 2017, pp. 19–38.

[40]

NuFHE (2019): NuFHE: a GPU-powered Torus FHE implementation. https://nufhe.readthedocs.io/en/latest/.

[41]

O. Ohrimenko, F. Schuster, C. Fournet, A. Mehta, S. Nowozin, K. Vaswani, M. Costa, Oblivious multi-party machine learning on trusted processors, in: 25th {USENIX} Security Symposium ({USENIX} Security 16), 2016, pp. 619–636.

[42]

P. Paillier, Public-key cryptosystems based on composite degree residuosity classes, in: International Conference on the Theory and Applications of Cryptographic Techniques, Springer, 1999, pp. 223–238.

[43]

R. Poddar, G. Ananthanarayanan, S. Setty, S. Volos, R.A. Popa, Visor: privacy-preserving video analytics as a cloud service, in: 29th {USENIX} Security Symposium ({USENIX} Security 20), 2020, pp. 1039–1056.

[44]

J.R. Sanchez Vicarte, B. Schreiber, R. Paccagnella, C.W. Fletcher, Game of threads: enabling asynchronous poisoning attacks, in: Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems, 2020, pp. 35–52.

[45]

SEAL, Microsoft SEAL (release 3.7), Microsoft Research, Redmond, WA, 2021, https://github.com/Microsoft/SEAL.

[46]

Shoup, V., et al., 2001. Ntl: A library for doing number theory.

[47]

N.P. Smart, F. Vercauteren, Fully homomorphic simd operations, Des. Codes Cryptogr. 71 (2014) 57–81.

[48]

Takeshita, J., McKechney, C., Pajak, J., Papadimitriou, A., Karl, R., Jung, T., 2021. Gps: Integration of graphene, palisade, and sgx for large-scale aggregations of distributed data. Cryptology ePrint Archive.

[49]

S. Volos, K. Vaswani, R. Bruno, Graviton: trusted execution environments on {GPUs}, in: 13th USENIX Symposium on Operating Systems Design and Implementation, in: OSDI, vol. 18, 2018, pp. 681–696.

[50]

Wagh, S., Gupta, D., Chandran, N., 2018. Securenn: Efficient and private neural network training. IACR Cryptol. ePrint Arch. 2018. p. 442.

[51]

Wang, W.; Jiang, Y.; Shen, Q.; Huang, W.; Chen, H.; Wang, S.; Wang, X.; Tang, H.; Chen, K.; Lauter, K.; et al. (2019): Toward scalable fully homomorphic encryption through light trusted computing assistance. arXiv preprint arXiv:1905.07766.

[52]

Z. Wang, P. Li, R. Hou, Z. Li, J. Cao, X. Wang, D. Meng, He-booster: an efficient polynomial arithmetic acceleration on gpus for fully homomorphic encryption, IEEE Trans. Parallel Distrib. Syst. 34 (2023) 1067–1081.

[53]

N. Weichbrodt, A. Kurmus, P. Pietzuch, R. Kapitza, Asyncshock: exploiting synchronisation bugs in intel sgx enclaves, in: European Symposium on Research in Computer Security, Springer, 2016, pp. 440–457.

[54]

D. Wu, J. Haven, Using homomorphic encryption for large scale statistical analysis, FHE-SI-Report, Univ. Stanford, Tech. Rep. TR-dwu4 2012.

[55]

H. Xiao, B. Biggio, G. Brown, G. Fumera, C. Eckert, F. Roli, Is feature selection secure against training data poisoning?, in: International Conference on Machine Learning, PMLR, 2015, pp. 1689–1698.

[56]

H. Xiao, Q. Zhang, Q. Pei, W. Shi, Privacy-preserving neural network inference framework via homomorphic encryption and sgx, in: 2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS), IEEE, 2021, pp. 751–761.

[57]

G. Xiong, K. Yan, X. Zhou, A distributed learning based sentiment analysis methods with web applications, World Wide Web 25 (2022) 1905–1922.

[58]

Y. Xu, W. Cui, M. Peinado, Controlled-channel attacks: deterministic side channels for untrusted operating systems, in: 2015 IEEE Symposium on Security and Privacy, IEEE, 2015, pp. 640–656.

[59]

P. Zhang, C. Song, H. Yin, D. Zou, E. Shi, H. Jin, Klotski: efficient obfuscated execution against controlled-channel attacks, in: Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems, 2020, pp. 1263–1276.

Index Terms

HT2ML: An efficient hybrid framework for privacy-preserving Machine Learning using HE and TEE
1. Security and privacy
  1. Security services
    1. Privacy-preserving protocols

Index terms have been assigned to the content through auto-classification.

Recommendations

Privacy-Preserving Certificateless Cloud Auditing with Multiple Users

Cloud auditing is one of the important processes to ensure the security and integrity of data in cloud storage. Implementing cloud auditing requires various cryptographic tools such as identity-based cryptography and its variant: certificateless ...
A Pairing-based Homomorphic Encryption Scheme for Multi-User Settings

A new method is presented to privately outsource computation of different users. As a significant cryptographic primitive in cloud computing, homomorphic encryption HE can evaluate on ciphertext directly without decryption, thus avoid information ...
An efficient and secure data sharing framework using homomorphic encryption in the cloud
Cloud-I '12: Proceedings of the 1st International Workshop on Cloud Intelligence

Due to cost-efficiency and less hands-on management, data owners are outsourcing their data to the cloud which can provide access to the data as a service. However, by outsourcing their data to the cloud, the data owners lose control over their data as ...

Comments

Information & Contributors

Information

Published In

cover image Computers and Security

Computers and Security Volume 135, Issue C

Dec 2023

755 pages

ISSN:0167-4048

Issue’s Table of Contents

The Author(s).

Publisher

Elsevier Advanced Technology Publications

United Kingdom

Publication History

Published: 10 January 2024

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Oct 2024

Other Metrics

View Author Metrics

Citations

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents