research-article

Privacy-Preserving Fair Learning of Support Vector Machine with Homomorphic Encryption

Authors:

Joohee LeeAuthors Info & Claims

WWW '22: Proceedings of the ACM Web Conference 2022

Pages 3572 - 3583

https://doi.org/10.1145/3485447.3512252

Published: 25 April 2022 Publication History

Abstract

Fair learning has received a lot of attention in recent years since machine learning models can be unfair in automated decision-making systems with respect to sensitive attributes such as gender, race, etc. However, to mitigate the discrimination on the sensitive attributes and train a fair model, most fair learning methods have required to get access to the sensitive attributes in training or validation phases. In this study, we propose a privacy-preserving training algorithm for a fair support vector machine classifier based on Homomorphic Encryption (HE), where the privacy of both sensitive information and model secrecy can be preserved. The expensive computational costs of HE can be significantly improved by protecting only the sensitive information, introducing refined formulation and low-rank approximation using shared eigenvectors. Through experiments on the synthetic and real-world data, we demonstrate the effectiveness of our algorithm in terms of accuracy and fairness and show that our method significantly outperforms other privacy-preserving solutions in terms of better trade-offs between accuracy and fairness. To the best of our knowledge, our algorithm is the first privacy-preserving fair learning algorithm using HE.

References

[1]

McKane Andrus, Elena Spitzer, Jeffrey Brown, and Alice Xiang. 2020. “What We Can’t Measure, We Can’t Understand”: Challenges to Demographic Data Procurement in the Pursuit of Fairness. arXiv preprint arXiv:2011.02282(2020).

[2]

Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine bias. ProPublica. See https://www. propublica. org/article/machine-bias-risk-assessments-in-criminal-sentencing(2016).

[3]

Anthony Barnett, Jay Santokhi, Michael Simpson, Nigel P Smart, Charlie Stainton-Bygrave, Srinivas Vivek, and Adrian Waller. 2017. Image Classification using non-linear Support Vector Machines on Encrypted Data.IACR Cryptol. ePrint Arch. 2017 (2017), 857.

[4]

Adi Ben-Israel. 1965. An iterative method for computing the generalized inverse of an arbitrary matrix. Math. Comp. (1965), 452–455.

[5]

Alex Beutel, Jilin Chen, Zhe Zhao, and Ed H Chi. 2017. Data decisions and theoretical implications when adversarially learning fair representations. arXiv preprint arXiv:1707.00075(2017).

[6]

Dan Biddle. 2006. Adverse impact and test validation: A practitioner’s guide to valid and defensible employment testing. Gower Publishing, Ltd.

[7]

Fabian Boemer, Rosario Cammarota, Daniel Demmler, Thomas Schneider, and Hossein Yalame. 2020. MP2ML: a mixed-protocol machine learning framework for private inference. In Proceedings of the 15th International Conference on Availability, Reliability and Security. 1–10.

Digital Library

[8]

Zvika Brakerski, Craig Gentry, and Vinod Vaikuntanathan. 2014. (Leveled) fully homomorphic encryption without bootstrapping. ACM Transactions on Computation Theory (TOCT) 6, 3 (2014), 1–36.

Digital Library

[9]

J Byun, J Lee, and S Park. 2021. Privacy-preserving evaluation for support vector clustering. Electronics Letters (2021).

[10]

Hui Cao, Takashi Naito, and Yoshiki Ninomiya. 2008. Approximate RBF kernel SVM and its applications in pedestrian classification. In The 1st International Workshop on Machine Learning for Vision-based Motion Analysis-MLVMA’08.

[11]

Hao Chen, Ilaria Chillotti, and Yongsoo Song. 2019. Improved Bootstrapping for Approximate Homomorphic Encryption. In Advances in Cryptology – EUROCRYPT 2019, Yuval Ishai and Vincent Rijmen (Eds.). Springer International Publishing, Cham, 34–54.

Digital Library

[12]

Hao Chen, Ran Gilad-Bachrach, Kyoohyung Han, Zhicong Huang, Amir Jalali, Kim Laine, and Kristin Lauter. 2018. Logistic regression over encrypted data from fully homomorphic encryption. BMC medical genomics 11, 4 (2018), 3–12.

[13]

Jung Hee Cheon, Kyoohyung Han, Andrey Kim, Miran Kim, and Yongsoo Song. 2018. Bootstrapping for approximate homomorphic encryption. In Annual International Conference on the Theory and Applications of Cryptographic Techniques. Springer, 360–384.

[14]

Jung Hee Cheon, Seungwan Hong, and Duhyeong Kim. 2020. Remark on the Security of CKKS Scheme in Practice. Cryptology ePrint Archive, Report 2020/1581. https://eprint.iacr.org/2020/1581.

[15]

Jung Hee Cheon, Andrey Kim, Miran Kim, and Yongsoo Song. 2017. Homomorphic encryption for arithmetic of approximate numbers. In International Conference on the Theory and Application of Cryptology and Information Security. Springer, 409–437.

[16]

Jung Hee Cheon, Dongwoo Kim, Duhyeong Kim, Hun Hee Lee, and Keewoo Lee. 2019. Numerical method for comparison on homomorphically encrypted numbers. In International Conference on the Theory and Application of Cryptology and Information Security. Springer, 415–445.

Digital Library

[17]

Jung Hee Cheon, Duhyeong Kim, and Jai Hyun Park. 2019. Towards a practical cluster analysis over encrypted data. In International Conference on Selected Areas in Cryptography. Springer, 227–249.

[18]

Alexandra Chouldechova. 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data 5, 2 (2017), 153–163.

[19]

Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. 2017. Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining. 797–806.

Digital Library

[20]

Elliot Creager, David Madras, Jörn-Henrik Jacobsen, Marissa Weis, Kevin Swersky, Toniann Pitassi, and Richard Zemel. 2019. Flexibly fair representation learning by disentanglement. In International Conference on Machine Learning. PMLR, 1436–1445.

[21]

Dheeru Dua and Casey Graff. 2017. UCI Machine Learning Repository. http://archive.ics.uci.edu/ml

[22]

Léo Ducas and Damien Stehlé. 2016. Sanitization of FHE Ciphertexts. In Advances in Cryptology – EUROCRYPT 2016, Marc Fischlin and Jean-Sébastien Coron (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 294–310.

[23]

Craig Gentry, Amit Sahai, and Brent Waters. 2013. Homomorphic encryption from learning with errors: Conceptually-simpler, asymptotically-faster, attribute-based. In Annual Cryptology Conference. Springer, 75–92.

[24]

Sara Hajian, Francesco Bonchi, and Carlos Castillo. 2016. Algorithmic bias: From discrimination discovery to fairness-aware data mining. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2125–2126.

Digital Library

[25]

Kyoohyung Han, Seungwan Hong, Jung Hee Cheon, and Daejun Park. 2019. Logistic regression on homomorphic encrypted data at scale. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 9466–9471.

Digital Library

[26]

Kyoohyung Han and Dohyeong Ki. 2020. Better Bootstrapping for Approximate Homomorphic Encryption. In Topics in Cryptology – CT-RSA 2020, Stanislaw Jarecki (Ed.). Springer International Publishing, Cham, 364–390.

Digital Library

[27]

Moritz Hardt, Eric Price, and Nati Srebro. 2016. Equality of Opportunity in Supervised Learning. In NIPS.

[28]

Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. 2018. Fairness without demographics in repeated loss minimization. In International Conference on Machine Learning. PMLR, 1929–1938.

[29]

Matthew Jagielski, Michael Kearns, Jieming Mao, Alina Oprea, Aaron Roth, Saeed Sharifi-Malvajerdi, and Jonathan Ullman. 2019. Differentially private fair learning. In International Conference on Machine Learning. PMLR, 3000–3008.

[30]

Niki Kilbertus, Adrià Gascón, Matt Kusner, Michael Veale, Krishna Gummadi, and Adrian Weller. 2018. Blind justice: Fairness with encrypted sensitive attributes. In International Conference on Machine Learning. PMLR, 2630–2639.

[31]

Miran Kim, Yongsoo Song, Shuang Wang, Yuhou Xia, and Xiaoqian Jiang. 2018. Secure logistic regression based on homomorphic encryption: Design and evaluation. JMIR medical informatics 6, 2 (2018), e19.

[32]

Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2016. Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807(2016).

[33]

Preethi Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, and Ed H Chi. 2020. Fairness without demographics through adversarially reweighted learning. arXiv preprint arXiv:2006.13114(2020).

[34]

Sungyoon Lee, Jaewook Lee, and Saerom Park. 2020. Lipschitz-certifiable training with a tight outer bound. Advances in Neural Information Processing Systems 33 (2020).

[35]

Baiyu Li and Daniele Micciancio. 2021. On the security of homomorphic encryption on approximate numbers. In Annual International Conference on the Theory and Applications of Cryptographic Techniques. Springer, 648–677.

Digital Library

[36]

Yehuda Lindell. 2016. How To Simulate It - A Tutorial on the Simulation Proof Technique. Cryptology ePrint Archive, Report 2016/046. https://eprint.iacr.org/2016/046.

[37]

David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. 2018. Learning adversarially fair and transferable representations. In International Conference on Machine Learning. PMLR, 3384–3393.

[38]

Hussein Mozannar, Mesrob Ohannessian, and Nathan Srebro. 2020. Fair learning with private demographic data. In International Conference on Machine Learning. PMLR, 7066–7075.

[39]

Saerom Park, Junyoung Byun, Joohee Lee, Jung Hee Cheon, and Jaewook Lee. 2020. HE-friendly algorithm for privacy-preserving SVM training. IEEE Access 8(2020), 57414–57425.

[40]

Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q Weinberger. 2017. On fairness and calibration. arXiv preprint arXiv:1709.02012(2017).

[41]

Carl Edward Rasmussen. 2003. Gaussian processes in machine learning. In Summer school on machine learning. Springer, 63–71.

[42]

Shahar Segal, Yossi Adi, Benny Pinkas, Carsten Baum, Chaya Ganesh, and Joseph Keshet. 2020. Fairness in the Eyes of the Data: Certifying Machine-Learning Models. arXiv preprint arXiv:2009.01534(2020).

[43]

Till Speicher, Hoda Heidari, Nina Grgic-Hlaca, Krishna P Gummadi, Adish Singla, Adrian Weller, and Muhammad Bilal Zafar. 2018. A unified approach to quantifying algorithmic unfairness: Measuring individual &group unfairness via inequality indices. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2239–2248.

Digital Library

[44]

Jacob Steinhardt, Pang Wei Koh, and Percy Liang. 2017. Certified defenses for data poisoning attacks. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 3520–3532.

[45]

Florian Tramèr, Fan Zhang, Ari Juels, Michael K. Reiter, and Thomas Ristenpart. 2016. Stealing Machine Learning Models via Prediction APIs. In 25th USENIX Security Symposium (USENIX Security 16). USENIX Association, Austin, TX, 601–618. https://www.usenix.org/conference/usenixsecurity16/technical-sessions/presentation/tramer

Digital Library

[46]

Michael Veale and Reuben Binns. 2017. Fairer machine learning in the real world: Mitigating discrimination without collecting sensitive data. Big Data & Society 4, 2 (2017), 2053951717743530.

[47]

Xinyuan Wu. 2007. A note on computational algorithm for the inverse of a square matrix. Applied mathematics and computation 187, 2 (2007), 962–964.

[48]

Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P Gummadi. 2017. Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In Proceedings of the 26th international conference on world wide web. 1171–1180.

Digital Library

[49]

Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez-Rodriguez, and Krishna P Gummadi. 2019. Fairness Constraints: A Flexible Approach for Fair Classification.J. Mach. Learn. Res. 20, 75 (2019), 1–42.

[50]

Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi. 2017. Fairness constraints: Mechanisms for fair classification. In Artificial Intelligence and Statistics. PMLR, 962–970.

[51]

Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. 2013. Learning fair representations. In International conference on machine learning. PMLR, 325–333.

[52]

Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, Wenxiao Wang, Bo Li, and Dawn Song. 2020. The secret revealer: Generative model-inversion attacks against deep neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 253–261.

Cited By

Hort MChen ZZhang JHarman MSarro F(2024)Bias Mitigation for Machine Learning Classifiers: A Comprehensive SurveyACM Journal on Responsible Computing10.1145/36313261:2(1-52)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3631326
Wang ZXia XXue MKhalil ILiwang MYi XChua TNgo CKa-Wei Lee RKumar RLauw H(2024)GEES: Enabling Location Privacy-Preserving Energy Saving in Multi-Access Edge ComputingProceedings of the ACM on Web Conference 202410.1145/3589334.3645329(2735-2746)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645329
Zhang HYuan XPan S(2024)Unraveling Privacy Risks of Individual Fairness in Graph Neural Networks2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00139(1712-1725)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00139
Show More Cited By

Index Terms

Privacy-Preserving Fair Learning of Support Vector Machine with Homomorphic Encryption

Index terms have been assigned to the content through auto-classification.

Recommendations

A secure and privacy-preserving word vector training scheme based on functional encryption with inner-product predicates
Abstract
With the extensive applications of machine learning, it has been witnessed that machine learning has been applied in various fields such as e-commerce, mobile data processing, health analytics and behavioral analytics etc. Word vector ...
GuardML: Efficient Privacy-Preserving Machine Learning Services Through Hybrid Homomorphic Encryption
SAC '24: Proceedings of the 39th ACM/SIGAPP Symposium on Applied Computing

Machine Learning (ML) has emerged as one of data science's most transformative and influential domains. However, the widespread adoption of ML introduces privacy-related concerns owing to the increasing number of malicious attacks targeting ML models. To ...
Deep Homeomorphic Data Encryption for Privacy Preserving Machine Learning
Abstract
Addressing privacy concerns is critical in smart manufacturing where sensitive data is used for machine learning. Data protection is essential to ensure model accuracy while upholding data privacy. Homeomorphic encryption, an algorithm for privacy-...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

3764 pages

ISBN:9781450390965

DOI:10.1145/3485447

Editors:
Frédérique Laforest
INSA Lyon, France
,
Raphaël Troncy
EURECOM, France
,
Elena Simperl
King’s College London, UK
,
Deepak Agarwal
Pinterest, USA
,
Aristides Gionis
KTH Royal Institute of Technology, Sweden
,
Ivan Herman
W3C / retired
,
Lionel Médini
Université Lyon 1, France

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

National Research Foundation of Korea(NRF)

Conference

WWW '22

Sponsor:

SIGWEB

WWW '22: The ACM Web Conference 2022

April 25 - 29, 2022

Virtual Event, Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
517
Total Downloads

Downloads (Last 12 months)180
Downloads (Last 6 weeks)6

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hort MChen ZZhang JHarman MSarro F(2024)Bias Mitigation for Machine Learning Classifiers: A Comprehensive SurveyACM Journal on Responsible Computing10.1145/36313261:2(1-52)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3631326
Wang ZXia XXue MKhalil ILiwang MYi XChua TNgo CKa-Wei Lee RKumar RLauw H(2024)GEES: Enabling Location Privacy-Preserving Energy Saving in Multi-Access Edge ComputingProceedings of the ACM on Web Conference 202410.1145/3589334.3645329(2735-2746)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645329
Zhang HYuan XPan S(2024)Unraveling Privacy Risks of Individual Fairness in Graph Neural Networks2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00139(1712-1725)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00139
Wang HChen TDing YWang YYang C(2024)Privacy-preserving multi-party logistic regression in cloud computingComputer Standards & Interfaces10.1016/j.csi.2024.10385790(103857)Online publication date: Aug-2024
https://doi.org/10.1016/j.csi.2024.103857
Falcetta ARoveri M(2024)EVAD: encrypted vibrational anomaly detection with homomorphic encryptionNeural Computing and Applications10.1007/s00521-024-09464-w36:13(7359-7372)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1007/s00521-024-09464-w
Pal ARanzato FUrban CZanella M(2024)Abstract Interpretation-Based Feature Importance for Support Vector MachinesVerification, Model Checking, and Abstract Interpretation10.1007/978-3-031-50524-9_2(27-49)Online publication date: 15-Jan-2024
https://dl.acm.org/doi/10.1007/978-3-031-50524-9_2
Wang XGu TBao XChang LLi L(2023)Individual fairness for local private graph neural networkKnowledge-Based Systems10.1016/j.knosys.2023.110490268:COnline publication date: 23-May-2023
https://dl.acm.org/doi/10.1016/j.knosys.2023.110490
Ko HByun JLee J(2023)A privacy-preserving robo-advisory system with the Black-Litterman portfolio model: A new framework and insights into investor behaviorJournal of International Financial Markets, Institutions and Money10.1016/j.intfin.2023.10187389(101873)Online publication date: Dec-2023
https://doi.org/10.1016/j.intfin.2023.101873
Park JChoi YByun JLee JPark S(2023)Efficient differentially private kernel support vector classifier for multi-class classificationInformation Sciences: an International Journal10.1016/j.ins.2022.10.075619:C(889-907)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1016/j.ins.2022.10.075
Byun JKo HLee J(2023)A Privacy-preserving mean–variance optimal portfolioFinance Research Letters10.1016/j.frl.2023.10379454(103794)Online publication date: Jun-2023
https://doi.org/10.1016/j.frl.2023.103794
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents