research-article

Privacy-Preserving Distributed Machine Learning Made Faster

Authors:

Xuan WangAuthors Info & Claims

SecTL '23: Proceedings of the 2023 Secure and Trustworthy Deep Learning Systems Workshop

Article No.: 2, Pages 1 - 14

https://doi.org/10.1145/3591197.3591306

Published: 10 July 2023 Publication History

Abstract

With the development of machine learning, it is difficult for a single server to process all the data. So machine learning tasks need to be spread across multiple servers, turning the centralized machine learning into a distributed one. Multi-key homomorphic encryption is one of the suitable candidates to solve the problem. However, the most recent result of the Multi-key homomorphic encryption scheme (MKTFHE) only supports the NAND gate. Although it is Turing complete, it requires efficient encapsulation of the NAND gate to further support mathematical calculation. This paper designs and implements a series of operations on positive and negative integers accurately. First, we design basic bootstrapped gates, the efficiency of which is times that the number of using NAND to build. Second, we construct practical k-bit complement mathematical operators based on our basic binary bootstrapped gates. The constructed created can perform addition, subtraction, multiplication, and division on both positive and negative integers. Finally, we demonstrated the generality of the designed operators by achieving a distributed privacy-preserving machine learning algorithm, i.e. linear regression with two different solutions. Experiments show that the consumption time of the operators built with our gate is about 50 ∼ 70% shorter than built directly with NAND gate and the iteration time of linear regression with our gates is 66.7% shorter than with NAND gate directly.

References

[1]

Yoshinori Aono, Takuya Hayashi, Le Trieu Phong, and Lihua Wang. 2016. Scalable and secure logistic regression via homomorphic encryption. In Proceedings of the Sixth ACM Conference on Data and Application Security and Privacy. 142–144.

Digital Library

[2]

Fabian Boemer, Yixing Lao, Rosario Cammarota, and Casimir Wierzynski. 2019. nGraph-HE: a graph compiler for deep learning on homomorphically encrypted data. In Proceedings of the 16th ACM International Conference on Computing Frontiers, CF 2019. 3–13.

Digital Library

[3]

Christina Boura, Nicolas Gama, and Mariya Georgieva. 2018. Chimera: a unified framework for B/FV, TFHE and HEAAN fully homomorphic encryption and predictions for deep learning. IACR Cryptol. ePrint Arch. 2018 (2018), 758.

[4]

Christina Boura, Nicolas Gama, Mariya Georgieva, and Dimitar Jetchev. 2020. CHIMERA: Combining Ring-LWE-based Fully Homomorphic Encryption Schemes. J. Math. Cryptol. 14, 1 (2020), 316–338.

[5]

Florian Bourse, Michele Minelli, Matthias Minihold, and Pascal Paillier. 2018. Fast Homomorphic Evaluation of Deep Discretized Neural Networks. In Advances in Cryptology - CRYPTO 2018(Lecture Notes in Computer Science, Vol. 10993). 483–512.

[6]

Zvika Brakerski, Craig Gentry, and Vinod Vaikuntanathan. 2014. (Leveled) Fully Homomorphic Encryption without Bootstrapping. ACM Trans. Comput. Theory 6, 3 (2014), 13:1–13:36.

Digital Library

[7]

Zvika Brakerski and Renen Perlman. 2016. Lattice-Based Fully Dynamic Multi-key FHE with Short Ciphertexts. In Advances in Cryptology - CRYPTO 2016(Lecture Notes in Computer Science, Vol. 9814). 190–213.

[8]

Hao Chen, Ilaria Chillotti, and Yongsoo Song. 2019. Multi-Key homomorphic encryption from TFHE. In Advances in Cryptology - ASIACRYPT 2019 - 25th International Conference on the Theory and Application of Cryptology and Information Security(Lecture Notes in Computer Science, Vol. 11922). 446–472.

[9]

Hao Chen, Wei Dai, Miran Kim, and Yongsoo Song. 2019. Efficient Multi-Key Homomorphic Encryption with Packed Ciphertexts with Application to Oblivious Neural Network Inference. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, CCS 2019. 395–412.

Digital Library

[10]

Hao Chen, Ran Gilad-Bachrach, Kyoohyung Han, Zhicong Huang, Amir Jalali, Kim Laine, and Kristin E. Lauter. 2018. Logistic regression over encrypted data from fully homomorphic encryption. IACR Cryptol. ePrint Arch. 2018 (2018), 462.

[11]

Jung Hee Cheon, Kyoohyung Han, Andrey Kim, Miran Kim, and Yongsoo Song. 2018. Bootstrapping for Approximate Homomorphic Encryption. In Advances in Cryptology - EUROCRYPT 2018(Lecture Notes in Computer Science, Vol. 10820). 360–384.

[12]

Jung Hee Cheon, Andrey Kim, Miran Kim, and Yong Soo Song. 2017. Homomorphic Encryption for Arithmetic of Approximate Numbers. In Advances in Cryptology - ASIACRYPT 2017(Lecture Notes in Computer Science, Vol. 10624). 409–437.

[13]

Jung Hee Cheon, Duhyeong Kim, Yongdai Kim, and Yongsoo Song. 2018. Ensemble Method for Privacy-Preserving Logistic Regression Based on Homomorphic Encryption. IEEE Access 6 (2018), 46938–46948.

[14]

Ilaria Chillotti, Nicolas Gama, Mariya Georgieva, and Malika Izabachène. 2016. Faster Fully Homomorphic Encryption: Bootstrapping in Less Than 0.1 Seconds. In Advances in Cryptology - ASIACRYPT 2016(Lecture Notes in Computer Science, Vol. 10031). 3–33.

[15]

Ilaria Chillotti, Nicolas Gama, Mariya Georgieva, and Malika Izabachène. 2017. Faster Packed Homomorphic Operations and Efficient Circuit Bootstrapping for TFHE. In Advances in Cryptology - ASIACRYPT 2017(Lecture Notes in Computer Science, Vol. 10624). Springer, 377–408.

[16]

Ilaria Chillotti, Nicolas Gama, Mariya Georgieva, and Malika Izabachène. 2020. TFHE: Fast Fully Homomorphic Encryption Over the Torus. J. Cryptol. 33, 1 (2020), 34–91.

Digital Library

[17]

Michael Clear and Ciaran McGoldrick. 2015. Multi-identity and multi-key leveled FHE from learning with errors. In Advances in Cryptology - CRYPTO 2015(Lecture Notes in Computer Science, Vol. 9216). 630–656.

[18]

Jack LH Crawford, Craig Gentry, Shai Halevi, Daniel Platt, and Victor Shoup. 2018. Doing real work with FHE: The case of logistic regression. In Proceedings of the 6th Workshop on Encrypted Computing & Applied Homomorphic Cryptography. 1–12.

Digital Library

[19]

Jack L. H. Crawford, Craig Gentry, Shai Halevi, Daniel Platt, and Victor Shoup. 2018. Doing Real Work with FHE: The Case of Logistic Regression. In Proceedings of the 6th Workshop on Encrypted Computing & Applied Homomorphic Cryptography, WAHC@CCS 2018. 1–12.

Digital Library

[20]

Roshan Dathathri, Olli Saarikivi, Hao Chen, Kim Laine, Kristin E. Lauter, Saeed Maleki, Madanlal Musuvathi, and Todd Mytkowicz. 2019. CHET: an optimizing compiler for fully-homomorphic neural-network inferencing. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2019. 142–156.

Digital Library

[21]

David Froelicher, Juan R Troncoso-Pastoriza, Apostolos Pyrgelis, Sinem Sav, Joao Sa Sousa, Jean-Philippe Bossuat, and Jean-Pierre Hubaux. 2020. Scalable privacy-preserving distributed learning. arXiv preprint arXiv:2005.09532 (2020).

[22]

Craig Gentry. 2009. Fully homomorphic encryption using ideal lattices. In Proceedings of the 41st Annual ACM Symposium on Theory of Computing, STOC 2009. 169–178.

Digital Library

[23]

Craig Gentry, Amit Sahai, and Brent Waters. 2013. Homomorphic Encryption from Learning with Errors: Conceptually-Simpler, Asymptotically-Faster, Attribute-Based. In Advances in Cryptology - CRYPTO 2013(Lecture Notes in Computer Science, Vol. 8042). 75–92.

[24]

Ran Gilad-Bachrach, Nathan Dowlin, Kim Laine, Kristin E. Lauter, Michael Naehrig, and John Wernsing. 2016. CryptoNets: Applying Neural Networks to Encrypted Data with High Throughput and Accuracy. In Proceedings of the 33nd International Conference on Machine Learning, ICML 2016(JMLR Workshop and Conference Proceedings, Vol. 48). 201–210.

[25]

Thore Graepel, Kristin E. Lauter, and Michael Naehrig. 2012. ML Confidential: Machine Learning on Encrypted Data. In Information Security and Cryptology - ICISC 2012, Vol. 7839. 1–21.

[26]

Kai Hwang. 1979. Global and modular two’s complement cellular array multipliers. IEEE Trans. on Computers 28, 4 (1979), 300–306.

Digital Library

[27]

Qi Jia, Linke Guo, Zhanpeng Jin, and Yuguang Fang. 2018. Preserving Model Privacy for Machine Learning in Distributed Systems. IEEE Transactions on Parallel and Distributed Systems 29, 8 (2018), 1808–1822. https://doi.org/10.1109/TPDS.2018.2809624

[28]

Andrey Kim, Yongsoo Song, Miran Kim, Keewoo Lee, and Jung Hee Cheon. 2018. Logistic regression model training based on the approximate homomorphic encryption. BMC medical genomics 11, 4 (2018), 23–31.

[29]

Miran Kim, Yongsoo Song, Shuang Wang, Yuhou Xia, and Xiaoqian Jiang. 2018. Secure Logistic Regression based on Homomorphic Encryption. IACR Cryptol. ePrint Arch. 2018 (2018), 74.

[30]

Miran Kim, Yongsoo Song, Shuang Wang, Yuhou Xia, Xiaoqian Jiang, 2018. Secure logistic regression based on homomorphic encryption: Design and evaluation. JMIR medical informatics 6, 2 (2018), e8805.

[31]

Jascha Kolberg, Pawel Drozdowski, Marta Gomez-Barrero, Christian Rathgeb, and Christoph Busch. 2020. Efficiency analysis of post-quantum-secure face template protection schemes based on homomorphic encryption. In BIOSIG 2020 - Proceedings of the 19th International Conference of the Biometrics Special Interest Group(LNI, Vol. P-306). 175–182.

[32]

Hyang-Sook Lee and Jeongeun Park. 2019. On the Security of Multikey Homomorphic Encryption. In Cryptography and Coding - 17th IMA International Conference, IMACC 2019(Lecture Notes in Computer Science, Vol. 11929). 236–251.

[33]

Adriana López-Alt, Eran Tromer, and Vinod Vaikuntanathan. 2012. On-the-fly multiparty computation on the cloud via multikey fully homomorphic encryption. In Proceedings of the 44th Symposium on Theory of Computing Conference, STOC 2012. 1219–1234.

Digital Library

[34]

Pratyay Mukherjee and Daniel Wichs. 2016. Two round multiparty computation via multi-key FHE. In Advances in Cryptology - EUROCRYPT 2016(Lecture Notes in Computer Science, Vol. 9666). 735–763.

[35]

Chris Peikert and Sina Shiehian. 2016. Multi-key FHE from LWE, Revisited. In Theory of Cryptography - 14th International Conference, TCC 2016-B(Lecture Notes in Computer Science, Vol. 9986). 217–238.

[36]

Jerome Saltzer and M Frans Kaashoek. 2009. Principles of computer system design: an introduction. Morgan Kaufmann.

[37]

Yifan Tian, Laurent Njilla, Jiawei Yuan, and Shucheng Yu. 2021. Low-Latency Privacy-Preserving Outsourcing of Deep Neural Network Inference. IEEE Internet Things J. 8, 5 (2021), 3300–3309.

[38]

Joost Verbraeken, Matthijs Wolting, Jonathan Katzy, Jeroen Kloppenburg, Tim Verbelen, and Jan S Rellermeyer. 2020. A survey on distributed machine learning. ACM Computing Surveys (CSUR) 53, 2 (2020), 1–33.

Digital Library

[39]

Haridimos T Vergos. 2007. An Efficient BIST Scheme for Non-Restoring Array Dividers. In 10th Euromicro Conference on Digital System Design Architectures, Methods and Tools (DSD 2007). IEEE, 664–667.

[40]

Zhi-Hua Zhou. 2021. Machine learning. Springer Nature.

Cited By

Petrean DPotolea R(2024)Random forest evaluation using multi-key homomorphic encryption and lookup tablesInternational Journal of Information Security10.1007/s10207-024-00823-123:3(2023-2041)Online publication date: 14-Mar-2024
https://dl.acm.org/doi/10.1007/s10207-024-00823-1
Wang HJiang ZZhao YYiu SYang PChen MTan ZJin B(2024)SFPDML: Securer and Faster Privacy-Preserving Distributed Machine Learning Based on MKTFHEMobile Internet Security10.1007/978-981-97-4465-7_7(94-108)Online publication date: 12-Jul-2024
https://doi.org/10.1007/978-981-97-4465-7_7
Archa AKartheeban K(2024)A Review on Privacy Enhanced Distributed ML Against Poisoning AttacksAI Applications in Cyber Security and Communication Networks10.1007/978-981-97-3973-8_11(173-186)Online publication date: 18-Sep-2024
https://doi.org/10.1007/978-981-97-3973-8_11

Recommendations

Multi-key fully homomorphic encryption from NTRU and (R)LWE with faster bootstrapping
Abstract
Multi-Key Fully Homomorphic Encryption (MKFHE) supports homomorphic evaluation on ciphertexts encrypted under different keys, guaranteeing data privacy in the case of multiple mutually untrusted parties. In ASIACRYPT 2019, Chen et al. ...
GuardML: Efficient Privacy-Preserving Machine Learning Services Through Hybrid Homomorphic Encryption
SAC '24: Proceedings of the 39th ACM/SIGAPP Symposium on Applied Computing

Machine Learning (ML) has emerged as one of data science's most transformative and influential domains. However, the widespread adoption of ML introduces privacy-related concerns owing to the increasing number of malicious attacks targeting ML models. To ...
Privacy-Preserving Fair Learning of Support Vector Machine with Homomorphic Encryption
WWW '22: Proceedings of the ACM Web Conference 2022

Fair learning has received a lot of attention in recent years since machine learning models can be unfair in automated decision-making systems with respect to sensitive attributes such as gender, race, etc. However, to mitigate the discrimination on the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SecTL '23: Proceedings of the 2023 Secure and Trustworthy Deep Learning Systems Workshop

July 2023

73 pages

ISBN:9798400701818

DOI:10.1145/3591197

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSAC: ACM Special Interest Group on Security, Audit, and Control

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 July 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ASIA CCS '23

Sponsor:

SIGSAC

ASIA CCS '23: ACM Asia Conference on Computer and Communications Security

July 10 - 14, 2023

VIC, Melbourne, Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
89
Total Downloads

Downloads (Last 12 months)50
Downloads (Last 6 weeks)3

Reflects downloads up to 14 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Petrean DPotolea R(2024)Random forest evaluation using multi-key homomorphic encryption and lookup tablesInternational Journal of Information Security10.1007/s10207-024-00823-123:3(2023-2041)Online publication date: 14-Mar-2024
https://dl.acm.org/doi/10.1007/s10207-024-00823-1
Wang HJiang ZZhao YYiu SYang PChen MTan ZJin B(2024)SFPDML: Securer and Faster Privacy-Preserving Distributed Machine Learning Based on MKTFHEMobile Internet Security10.1007/978-981-97-4465-7_7(94-108)Online publication date: 12-Jul-2024
https://doi.org/10.1007/978-981-97-4465-7_7
Archa AKartheeban K(2024)A Review on Privacy Enhanced Distributed ML Against Poisoning AttacksAI Applications in Cyber Security and Communication Networks10.1007/978-981-97-3973-8_11(173-186)Online publication date: 18-Sep-2024
https://doi.org/10.1007/978-981-97-3973-8_11

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents