research-article

Level Up: Private Non-Interactive Decision Tree Evaluation using Levelled Homomorphic Encryption

Authors:

Rasoul Akhavan Mahdavi,

Dimitry Linkov,

Florian KerschbaumAuthors Info & Claims

CCS '23: Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security

Pages 2945 - 2958

https://doi.org/10.1145/3576915.3623095

Published: 21 November 2023 Publication History

Abstract

As machine learning as a service continues gaining popularity, concerns about privacy and intellectual property arise. Users often hesitate to disclose their private information to obtain a service, while service providers aim to protect their proprietary models. Decision trees, a widely used machine learning model, are favoured for their simplicity, interpretability, and ease of training. In this context, Private Decision Tree Evaluation (PDTE) enables a server holding a private decision tree to provide predictions based on a client's private attributes. The protocol is such that the server learns nothing about the client's private attributes. Similarly, the client learns nothing about the server's model besides the prediction and some hyperparameters.

In this paper, we propose two novel non-interactive PDTE protocols, XXCMP-PDTE and RCC-PDTE, based on two new non-interactive comparison protocols, XXCMP and RCC. Our evaluation of these comparison operators demonstrates that our proposed constructions can efficiently evaluate high-precision numbers. Specifically, RCC can compare 32-bit numbers in under 10 milliseconds.

We assess our proposed PDTE protocols on decision trees trained over UCI datasets and compare our results with existing work in the field. Moreover, we evaluate synthetic decision trees to showcase scalability, revealing that RCC-PDTE can evaluate a decision tree with over 1000 nodes and 16 bits of precision in under 2 seconds. In contrast, the current state-of-the-art requires over 10 seconds to evaluate such a tree with only 11 bits of precision.

References

[1]

[n. d.]. Amazon Machine Learning. https://aws.amazon.com/machine-learning. Accessed May 2, 2023.

[2]

[n. d.]. Azure Machine Learning. https://azure.microsoft.com/products/machine- learning/. Accessed May 2, 2023.

[3]

[n. d.]. BigML. https://bigml.com/. Accessed May 2, 2023.

[4]

[n. d.]. Google Cloud Vertex AI. https://cloud.google.com/vertex-ai. Accessed May 2, 2023.

[5]

Ehud Aharoni, Allon Adir, Moran Baruch, Nir Drucker, Gilad Ezov, Ariel Farkash, Lev Greenberg, Ramy Masalha, Guy Moshkowich, Dov Murik, Hayim Shaul, and Omri Soceanu. 2023. HeLayers: A Tile Tensors Framework for Large Neural Networks on Encrypted Data. Privacy Enhancing Technology Symposium (PETs) 2023 (2023). https://petsymposium.org/2023/paperlist.php

[6]

Adi Akavia, Max Leibovich, Yehezkel S. Resheff, Roey Ron, Moni Shahar, and Margarita Vald. 2022. Privacy-Preserving Decision Trees Training and Prediction. ACM Trans. Priv. Secur., Vol. 25, 3, Article 24 (may 2022), 30 pages. https://doi.org/10.1145/3517197

Digital Library

[7]

Sebastian Angel, Hao Chen, Kim Laine, and Srinath Setty. 2018. PIR with Compressed Queries and Amortized Query Processing. In 2018 IEEE Symposium on Security and Privacy (SP). 962--979. https://doi.org/10.1109/SP.2018.00062

[8]

Sofiane Azogagh, Victor Delfour, Sébastien Gambs, and Marc-Olivier Killijian. 2022. PROBONITE: PRivate One-Branch-Only Non-Interactive Decision Tree Evaluation. In Proceedings of the 10th Workshop on Encrypted Computing & Applied Homomorphic Cryptography (Los Angeles, CA, USA) (WAHC'22). Association for Computing Machinery, New York, NY, USA, 23--33. https://doi.org/10.1145/3560827.3563377

Digital Library

[9]

Jianli Bai, Xiangfu Song, Shujie Cui, Ee-Chien Chang, and Giovanni Russello. 2022. Scalable Private Decision Tree Evaluation with Sublinear Communication. In Proceedings of the 2022 ACM on Asia Conference on Computer and Communications Security (Nagasaki, Japan) (ASIA CCS '22). Association for Computing Machinery, New York, NY, USA, 843--857. https://doi.org/10.1145/3488932.3517413

Digital Library

[10]

Raphael Bost, Raluca Ada Popa, Stephen Tu, and Shafi Goldwasser. 2014. Machine Learning Classification over Encrypted Data. Cryptology ePrint Archive (2014).

[11]

Zvika Brakerski, Craig Gentry, and Vinod Vaikuntanathan. 2012. (Leveled) Fully Homomorphic Encryption without Bootstrapping. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference (Cambridge, Massachusetts) (ITCS '12). Association for Computing Machinery, New York, NY, USA, 309--325. https://doi.org/10.1145/2090236.2090262

Digital Library

[12]

Justin Brickell, Donald E. Porter, Vitaly Shmatikov, and Emmett Witchel. 2007. Privacy-Preserving Remote Diagnostics. Association for Computing Machinery, New York, NY, USA, 498--507. https://doi.org/10.1145/1315245.1315307

Digital Library

[13]

Hao Chen, Ilaria Chillotti, and Ling Ren. 2019. Onion Ring ORAM: Efficient Constant Bandwidth Oblivious RAM from (Leveled) TFHE. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security (CCS '19). Association for Computing Machinery, New York, NY, USA, 345--360. https://doi.org/10.1145/3319535.3354226

Digital Library

[14]

Ilaria Chillotti, Nicolas Gama, Mariya Georgieva, and Malika Izabachène. 2020. TFHE: Fast Fully Homomorphic Encryption over the Torus. Journal of Cryptology, Vol. 33, 1 (2020), 34--91.

Digital Library

[15]

Kelong Cong, Debajyoti Das, Jeongeun Park, and Hilder V.L. Pereira. 2022. SortingHat: Efficient Private Decision Tree Evaluation via Homomorphic Encryption and Transciphering. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security (Los Angeles, CA, USA) (CCS '22 ). Association for Computing Machinery, New York, NY, USA, 563--577. https://doi.org/10.1145/3548606.3560702

Digital Library

[16]

Dheeru Dua and Casey Graff. 2017. UCI Machine Learning Repository. http://archive.ics.uci.edu/ml

[17]

Léo Ducas and Daniele Micciancio. 2015. FHEW: bootstrapping homomorphic encryption in less than a second. In Advances in Cryptology-EUROCRYPT 2015: 34th Annual International Conference on the Theory and Applications of Cryptographic Techniques, Sofia, Bulgaria, April 26-30, 2015, Proceedings, Part I 34. Springer, 617--640.

[18]

Junfeng Fan and Frederik Vercauteren. 2012. Somewhat Practical Fully Homomorphic Encryption. Proceedings of the 15th international conference on Practice and Theory in Public Key Cryptography, Vol. 2012 (2012), 1--16. https://eprint.iacr.org/2012/144

[19]

Yidi Hao, Baodong Qin, and Yitian Sun. 2023. Privacy-Preserving Decision-Tree Evaluation with Low Complexity for Communication. Sensors, Vol. 23, 5 (2023), 2624.

[20]

Trevor Hastie, Robert Tibshirani, Jerome H Friedman, and Jerome H Friedman. 2009. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Vol. 2. Springer.

[21]

Ilia Iliashenko and Vincent Zucca. 2021. Faster Homomorphic Comparison Operations for BGV and BFV. Proceedings on Privacy Enhancing Technologies, Vol. 2021, 3 (2021), 246--264. https://doi.org/10.2478/popets-2021-0046

[22]

Mika Juuti, Sebastian Szyller, Samuel Marchal, and N Asokan. 2019. PRADA: protecting against DNN model stealing attacks. In 2019 IEEE European Symposium on Security and Privacy (EuroS&P). IEEE, 512--527.

[23]

Manish Kesarwani, Bhaskar Mukhoty, Vijay Arya, and Sameep Mehta. 2018. Model Extraction Warning in MLaaS Paradigm. In Proceedings of the 34th Annual Computer Security Applications Conference (San Juan, PR, USA) (ACSAC '18). Association for Computing Machinery, New York, NY, USA, 371--380. https://doi.org/10.1145/3274694.3274740

Digital Library

[24]

Aggelos Kiayias, Stavros Papadopoulos, Nikos Triandopoulos, and Thomas Zacharias. 2013. Delegatable Pseudorandom Functions and Applications. In Proceedings of the 2013 ACM SIGSAC Conference on Computer and Communications Security (Berlin, Germany) (CCS '13 ). Association for Computing Machinery, New York, NY, USA, 669--684. https://doi.org/10.1145/2508859.2516668

Digital Library

[25]

Ágnes Kiss, Masoud Naderpour, Jian Liu, N Asokan, and Thomas Schneider. 2019. SoK: Modular and Efficient Private Decision Tree Evaluation. Proceedings on Privacy Enhancing Technologies, Vol. 2 (2019), 187--208.

[26]

Hsiao-Ying Lin and Wen-Guey Tzeng. 2005. An Efficient Solution to the Millionaires' Problem Based on Homomorphic Encryption. In Proceedings of the Third International Conference on Applied Cryptography and Network Security (New York, NY) (ACNS'05). Springer-Verlag, Berlin, Heidelberg, 456--466. https://doi.org/10.1007/11496137_31

Digital Library

[27]

Yehuda Lindell and Benny Pinkas. 2000. Privacy preserving data mining. In Advances in Cryptology-CRYPTO 2000: 20th Annual International Cryptology Conference Santa Barbara, California, USA, August 20-24, 2000 Proceedings. Springer, 36--54.

[28]

Wen-jie Lu, Zhicong Huang, Qizhi Zhang, Yuchen Wang, and Cheng Hong. 2023. Squirrel: A Scalable Secure Two-Party Computation Framework for Training Gradient Boosting Decision Tree. USENIX Security Symposium (2023).

[29]

Wen-jie Lu, Jun-jie Zhou, and Jun Sakuma. 2018. Non-Interactive and Output Expressive Private Comparison from Homomorphic Encryption. In Proceedings of the 2018 on Asia Conference on Computer and Communications Security (Incheon, Republic of Korea) (ASIACCS '18). Association for Computing Machinery, New York, NY, USA, 67--74. https://doi.org/10.1145/3196494.3196503

Digital Library

[30]

Rasoul Akhavan Mahdavi and Florian Kerschbaum. 2022. Constant-weight PIR: Single-round Keyword PIR via Constant-weight Equality Operators. In 31st USENIX Security Symposium (USENIX Security 22). USENIX Association, Boston, MA, 1723--1740. https://www.usenix.org/conference/usenixsecurity22/presentation/mahdavi

[31]

Arthur Meyre, Benoit Chevallier-Mames, Jordan Frery, Andrei Stoian, Roman Bredehoft, Luis Montero, and Celia Kherfallah. 2022. Concrete-ML: a Privacy-Preserving Machine Learning Library using Fully Homomorphic Encryption for Data Scientists. https://github.com/zama-ai/concrete-ml.

[32]

Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. 2017. Practical Black-Box Attacks against Machine Learning. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security (Abu Dhabi, United Arab Emirates) (ASIA CCS '17). Association for Computing Machinery, New York, NY, USA, 506--519. https://doi.org/10.1145/3052973.3053009

Digital Library

[33]

J. Ross Quinlan. 1986. Induction of Decision Trees. Machine learning, Vol. 1 (1986), 81--106.

[34]

Elaine Shi, John Bethencourt, T-H. Hubert Chan, Dawn Song, and Adrian Perrig. 2007. Multi-Dimensional Range Query over Encrypted Data. In 2007 IEEE Symposium on Security and Privacy (SP '07). Oakland, California, USA, 350--364. https://doi.org/10.1109/SP.2007.29

Digital Library

[35]

Elaine Shi, John Bethencourt, T-H. Hubert Chan, Dawn Song, and Adrian Perrig. 2007. Multi-Dimensional Range Query over Encrypted Data. In 2007 IEEE Symposium on Security and Privacy (SP '07). Oakland, California, USA, 350?364. https://doi.org/10.1109/SP.2007.29

Digital Library

[36]

R. Shokri, M. Stronati, C. Song, and V. Shmatikov. 2017. Membership Inference Attacks Against Machine Learning Models. In 2017 IEEE Symposium on Security and Privacy (SP). IEEE Computer Society, Los Alamitos, CA, USA, 3--18. https://doi.org/10.1109/SP.2017.41

[37]

Raymond KH Tai, Jack PK Ma, Yongjun Zhao, and Sherman SM Chow. 2017. Privacy-Preserving Decision Trees Evaluation via Linear Functions. In Computer Security-ESORICS 2017: 22nd European Symposium on Research in Computer Security, Oslo, Norway, September 11-15, 2017, Proceedings, Part II 22. Springer, 494--512.

[38]

Florian Tramèr, Fan Zhang, Ari Juels, Michael K. Reiter, and Thomas Ristenpart. 2016. Stealing Machine Learning Models via Prediction APIs. In Proceedings of the 25th USENIX Conference on Security Symposium (Austin, TX, USA) (SEC'16). USENIX Association, USA, 601--618.

Digital Library

[39]

Anselme Tueno, Yordan Boev, and Florian Kerschbaum. 2019. Non-Interactive Private Decision Tree Evaluation. In Database Security. Springer International Publishing, 174--194.

[40]

Anselme Tueno, Florian Kerschbaum, Stefan Katzenbeisser, and Privacy Enhancing Technologies Symposium. 2019-01-01. Private Evaluation of Decision Trees using Sublinear Cost. Proceedings on Privacy Enhancing Technologies, Vol. 2019, 1 (2019-01-01).

[41]

Yuncheng Wu, Shaofeng Cai, Xiaokui Xiao, Gang Chen, and Beng Chin Ooi. 2020. Privacy Preserving Vertical Federated Learning for Tree-based Models. Proceedings of the VLDB Endowment, Vol. 13, 11 (2020).

Digital Library

[42]

Wenting Zheng, Ryan Deng, Weikeng Chen, Raluca Ada Popa, Aurojit Panda, and Ion Stoica. 2021. Cerebro: A Platform for Multi-Party Cryptographic Collaborative Learning. In 30th USENIX Security Symposium (USENIX Security 21). USENIX Association, Online, 2723--2740. https://www.usenix.org/conference/usenixsecurity21/presentation/zheng.

Cited By

Cong KKang JNicolas GPark J(2024)Faster Private Decision Tree Evaluation for Batched Input from Homomorphic EncryptionSecurity and Cryptography for Networks10.1007/978-3-031-71073-5_1(3-23)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1007/978-3-031-71073-5_1
Shin HChoi JLee DKim KLee Y(2024)Fully Homomorphic Training and Inference on Binary Decision Tree and Random ForestComputer Security – ESORICS 202410.1007/978-3-031-70896-1_11(217-237)Online publication date: 6-Sep-2024
https://doi.org/10.1007/978-3-031-70896-1_11

Index Terms

Level Up: Private Non-Interactive Decision Tree Evaluation using Levelled Homomorphic Encryption
1. Security and privacy
  1. Cryptography
  2. Security services
    1. Privacy-preserving protocols

Recommendations

Privacy-preserving outsourcing decision tree evaluation from homomorphic encryption
Abstract
A decision tree is a common algorithm in machine learning, which performs classification prediction based on a tree structure. In real-world scenarios, input attribute values may be sensitive and tree models are usually private, thus ...
Private Decision Tree Evaluation with Constant Rounds via (Only) Fair SS-4PC
Information Security and Privacy
Abstract
Multiparty computation (MPC) is a cryptographic method that enables a set of parties to compute an arbitrary joint function of the private inputs of all parties and does not reveal any information other than the output. MPC based on a secret ...
Private Decision Tree Evaluation with Constant Rounds via (Only) SS-3PC over Ring
Provable and Practical Security
Abstract
Secure computation is the technology that computes an arbitrary function represented as a circuit without revealing input values. Typical technologies related to secure computation are secure multiparty computation (MPC) that uses secret sharing (...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CCS '23: Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security

November 2023

3722 pages

ISBN:9798400700507

DOI:10.1145/3576915

General Chairs:
Weizhi Meng
Technical University of Denmark
,
Christian D. Jensen
Technical University of Denmark
,
Program Chairs:
Cas Cremers
CISPA Helmholtz Center for Information Security
,
Engin Kirda
Khoury College of Computer Sciences

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSAC: ACM Special Interest Group on Security, Audit, and Control

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 November 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CCS '23

Sponsor:

SIGSAC

CCS '23: ACM SIGSAC Conference on Computer and Communications Security

November 26 - 30, 2023

Copenhagen, Denmark

Acceptance Rates

Overall Acceptance Rate 1,261 of 6,999 submissions, 18%

Upcoming Conference

CCS '24

Sponsor:
sigsac

ACM SIGSAC Conference on Computer and Communications Security

October 14 - 18, 2024

Salt Lake City , UT , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
389
Total Downloads

Downloads (Last 12 months)389
Downloads (Last 6 weeks)36

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cong KKang JNicolas GPark J(2024)Faster Private Decision Tree Evaluation for Batched Input from Homomorphic EncryptionSecurity and Cryptography for Networks10.1007/978-3-031-71073-5_1(3-23)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1007/978-3-031-71073-5_1
Shin HChoi JLee DKim KLee Y(2024)Fully Homomorphic Training and Inference on Binary Decision Tree and Random ForestComputer Security – ESORICS 202410.1007/978-3-031-70896-1_11(217-237)Online publication date: 6-Sep-2024
https://doi.org/10.1007/978-3-031-70896-1_11

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents