research-article

On significance of the least significant bits for differential privacy

Author:

Ilya MironovAuthors Info & Claims

CCS '12: Proceedings of the 2012 ACM conference on Computer and communications security

Pages 650 - 661

https://doi.org/10.1145/2382196.2382264

Published: 16 October 2012 Publication History

Abstract

We describe a new type of vulnerability present in many implementations of differentially private mechanisms. In particular, all four publicly available general purpose systems for differentially private computations are susceptible to our attack.

The vulnerability is based on irregularities of floating-point implementations of the privacy-preserving Laplacian mechanism. Unlike its mathematical abstraction, the textbook sampling procedure results in a porous distribution over double-precision numbers that allows one to breach differential privacy with just a few queries into the mechanism.

We propose a mitigating strategy and prove that it satisfies differential privacy under some mild assumptions on available implementation of floating-point arithmetic.

References

[1]

R. Bhaskar, A. Bhowmick, V. Goyal, S. Laxman, and A. Thakurta. Noiseless database privacy. In D. H. Lee and X. Wang, editors, Advances in Cryptology--ASIACRYPT 2011, volume 7073 of Lecture Notes in Computer Science, pages 215--232. Springer, 2011.

Digital Library

[2]

J. Blocki, A. Blum, A. Datta, and O. Sheffet. The Johnson-Lindenstrauss transform itself preserves differential privacy. In 53rd Symposium on Foundations of Computer Science (FOCS 2012). IEEE Computer Society, 2012. To appear.

Digital Library

[3]

K. Chaudhuri and C. Monteleoni. Privacy-preserving logistic regression. In D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, editors, Advances in Neural Information Processing Systems 21 (NIPS), pages 289--296. Curran Associates, Inc., 2008.

[4]

Correctly rounded mathematical library. http://lipforge.ens-lyon.fr/www/crlibm/.

[5]

G. Danezis, M. Kohlweiss, and A. Rial. Differentially private billing with rebates. In Proceedings of the 13th international conference on Information hiding, IH'11, pages 148--162, Berlin, Heidelberg, 2011. Springer-Verlag.

Digital Library

[6]

F. de Dinechin, C. Q. Lauter, and J.-M. Muller. Fast and correctly rounded logarithms in double-precision. Theoretical Informatics and Applications, 41(1):85--102, 2007.

[7]

Y. Dodis, A. López-Alt, I. Mironov, and S. Vadhan. Differential privacy with imperfect randomness. In Advances in Cryptology--CRYPTO 2012, 2012. To appear. Full version http://eprint.iacr.org/2012/435.

Digital Library

[8]

Y. Dodis, S. J. Ong, M. Prabhakaran, and A. Sahai. On the (im)possibility of cryptography with imperfect randomness. In FOCS '04, pages 196--205. IEEE Computer Society, 2004.

Digital Library

[9]

C. Dwork. Differential privacy. Invited talk. In M. Bugliesi, B. Preneel, V. Sassone, and I. Wegener, editors, Automata, Languages and Programming--ICALP (2), volume 4052 of Lecture Notes in Computer Science, pages 1--12. Springer, 2006.

Digital Library

[10]

C. Dwork. A firm foundation for private data analysis. Commun. ACM, 54(1):86--95, 2011.

Digital Library

[11]

C. Dwork, K. Kenthapadi, F. McSherry, I. Mironov, and M. Naor. Our data, ourselves: Privacy via distributed noise generation. In S. Vaudenay, editor, Advances in Cryptology--EUROCRYPT 2006, volume 4004 of Lecture Notes in Computer Science, pages 486--503. Springer, 2006.

Digital Library

[12]

C. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating noise to sensitivity in private data analysis. In S. Halevi and T. Rabin, editors, Theory of Cryptography Conference--TCC 2006, volume 3876 of Lecture Notes in Computer Science, pages 265--284. Springer, 2006.

Digital Library

[13]

C. Dwork and K. Nissim. Privacy-preserving datamining on vertically partitioned databases. In M. K. Franklin, editor, Advances in Cryptology--CRYPTO 2004, volume 3152 of Lecture Notes in Computer Science, pages 528--544. Springer, 2004.

[14]

T. Evans, L. Zayatz, and J. Slanta. Using noise for disclosure limitation of establishment tabular data. J. of Official Statistics, 14(4):537--551, 1998.

[15]

A. Ghosh, T. Roughgarden, and M. Sundararajan. Universally utility-maximizing privacy mechanisms. In M. Mitzenmacher, editor, Proceedings of the 41st Annual ACM Symposium on Theory of Computing, STOC 2009, pages 351--360. ACM, 2009.

Digital Library

[16]

V. D. Gligor. A guide to understanding covert channel analysis of trusted systems. Technical Report NCSC-TG-030, National Computer Security Center, Nov. 1993.

[17]

D. Goldberg. What every computer scientist should know about floating-point arithmetic. ACM Computing Surveys, 23(1):5--48, Mar. 1991.

Digital Library

[18]

A. Haeberlen, B. C. Pierce, and A. Narayan. Differential privacy under fire. In USENIX Security Symposium. USENIX Association, 2011.

Digital Library

[19]

M. Hardt, K. Ligett, and F. McSherry. A simple and practical algorithm for differentially private data release. CoRR, abs/1012.4763, 2010.

[20]

N. J. Higham. Accuracy and Stability of Numerical Algorithms. SIAM: Society for Industrial and Applied Mathematics, 2nd edition, Aug. 2002.

Digital Library

[21]

IEEE standard for floating-point arithmetic. Technical report, Microprocessor Standards Committee of the IEEE Computer Society, 3 Park Avenue, New York, NY 10016--5997, USA, Aug. 2008.

[22]

W. Kahan. Pracniques: further remarks on reducing truncation errors. Commun. ACM, 8(1):40, Jan. 1965.

Digital Library

[23]

D. E. Knuth. Seminumerical Algorithms, volume 2 of The Art of Computer Programming. Addison-Wesley, third edition, 1997.

Digital Library

[24]

K. Kursawe, G. Danezis, and M. Kohlweiss. Privacy-friendly aggregation for the smart-grid. In S. Fischer-Hübner and N. Hopper, editors, PETS, volume 6794 of Lecture Notes in Computer Science, pages 175--191. Springer, 2011.

Digital Library

[25]

B. W. Lampson. A note on the confinement problem. Communications of the ACM, 16(10):613--615, Oct. 1973.

Digital Library

[26]

P. L'Ecuyer. SSJ: Stochastic simulation in Java. http://www.iro.umontreal.ca/ simardr/ssj/indexe.html.

[27]

C. Li, M. Hay, V. Rastogi, G. Miklau, and A. McGregor. Optimizing linear counting queries under differential privacy. In J. Paredaens and D. V. Gucht, editors, Symposium on Principles of Database Systems, PODS 2010, pages 123--134. ACM, 2010.

Digital Library

[28]

A. Machanavajjhala, D. Kifer, J. M. Abowd, J. Gehrke, and L. Vilhuber. Privacy: Theory meets practice on the map. In G. Alonso, J. A. Blakeley, and A. L. P. Chen, editors, Proceedings of the 24th International Conference on Data Engineering, ICDE 2008, pages 277--286. IEEE Computer Society, 2008.

Digital Library

[29]

J. L. McInnes and B. Pinkas. On the impossibility of private key cryptography with weakly random keys. In A. Menezes and S. A. Vanstone, editors, Advances in Cryptology--CRYPTO '90, volume 537 of Lecture Notes in Computer Science, pages 421--435. Springer, 1991.

Digital Library

[30]

F. McSherry. Privacy integrated queries: an extensible platform for privacy-preserving data analysis. In U. Çetintemel, S. B. Zdonik, D. Kossmann, and N. Tatbul, editors, Proceedings of the ACM SIGMOD International Conference on Management of Data, SIGMOD 2009, pages 19--30. ACM, 2009.

Digital Library

[31]

F. McSherry. Privacy integrated queries: an extensible platform for privacy-preserving data analysis. Commun. ACM, 53(9):89--97, 2010.

Digital Library

[32]

F. McSherry and R. Mahajan. Differentially-private network trace analysis. In S. Kalyanaraman, V. N. Padmanabhan, K. K. Ramakrishnan, R. Shorey, and G. M. Voelker, editors, SIGCOMM, pages 123--134. ACM, 2010.

Digital Library

[33]

F. McSherry and I. Mironov. Differentially private recommender systems: Building privacy into the Netflix Prize contenders. In J. F. Elder, IV, F. Fogelman-Soulié, P. A. Flach, and M. J. Zaki, editors, Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining--KDD 2009, pages 627--636. ACM, June 2009.

Digital Library

[34]

P. Mohan, A. G. Thakurta, E. Shi, D. Song, and D. E. Culler. GUPT: Privacy preserving data analysis made easy. In Proceedings of the 38th SIGMOD international conference on Management of data, SIGMOD '12, New York, NY, USA, 2012. ACM.

Digital Library

[35]

J.-M. Muller, N. Brisebarre, F. de Dinechin, C.-P. Jeannerod, V. Lefèvre, G. Melquiond, N. Revol, D. Stehlé, and S. Torres. Handbook of Floating-Point Arithmetic. Birkhauser Boston, 2010.

Digital Library

[36]

K. Nissim, S. Raskhodnikova, and A. Smith. Smooth sensitivity and sampling in private data analysis. In D. S. Johnson and U. Feige, editors, Proceedings of the 39th Annual ACM Symposium on Theory of Computing (STOC), pages 75--84. ACM, 2007.

Digital Library

[37]

W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery. Numerical Recipes: The Art of Scientific Computing. Cambridge University Press, New York, NY, USA, third edition, 2007.

Digital Library

[38]

J. Reed and B. C. Pierce. Distance makes the types grow stronger: a calculus for differential privacy. In P. Hudak and S. Weirich, editors, Proceeding of the 15th ACM SIGPLAN international conference on Functional programming, ICFP 2010, pages 157--168. ACM, 2010.

Digital Library

[39]

I. Roy, S. T. V. Setty, A. Kilzer, V. Shmatikov, and E. Witchel. Airavat: Security and privacy for MapReduce. In Proceedings of the 7th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2010, pages 297--312. USENIX Association, 2010.

Digital Library

[40]

M. Santha and U. V. Vazirani. Generating quasi-random sequences from semi-random sources. J. Comput. Syst. Sci., 33(1):75--87, 1986.

Digital Library

Cited By

Goualard F(2024)MicroFloatingPoints.jl: providing very small IEEE 754-compliant floating-point typesJournal of Open Source Software10.21105/joss.070509:101(7050)Online publication date: Oct-2024
https://doi.org/10.21105/joss.07050
Cho HFroelicher DDokmai NNandi ASadhuka SHong MBerger B(2024)Privacy-Enhancing Technologies in Biomedical Data ScienceAnnual Review of Biomedical Data Science10.1146/annurev-biodatasci-120423-1201077:1(317-343)Online publication date: 23-Aug-2024
https://doi.org/10.1146/annurev-biodatasci-120423-120107
Song PSarathy JShoemate MVadhan S(2024)"I inherently just trust that it works": Investigating Mental Models of Open-Source Libraries for Differential PrivacyProceedings of the ACM on Human-Computer Interaction10.1145/36870118:CSCW2(1-39)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3687011
Show More Cited By

Index Terms

On significance of the least significant bits for differential privacy
1. Security and privacy
  1. Database and storage security
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Database theory
      1. Theory of database privacy and security

Recommendations

An Emerging Strategy for Privacy Preserving Databases: Differential Privacy
HCI for Cybersecurity, Privacy and Trust
Abstract
Data De-identification and Differential Privacy are two possible approaches for providing data security and user privacy. Data de-identification is the process where the personal identifiable information of individuals is extracted to create ...
A Novel Differential Privacy Approach that Enhances Classification Accuracy
C3S2E '16: Proceedings of the Ninth International C* Conference on Computer Science & Software Engineering

In the recent past, there has been a tremendous increase of large repositories of data, examples being in healthcare data, consumer data from retailers, and airline passenger data. These data are continually being shared with interested parties, either ...
Differential Privacy: Now it's Getting Personal
POPL '15: Proceedings of the 42nd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages

Differential privacy provides a way to get useful information about sensitive data without revealing much about any one individual. It enjoys many nice compositionality properties not shared by other approaches to privacy, including, in particular, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CCS '12: Proceedings of the 2012 ACM conference on Computer and communications security

October 2012

1088 pages

ISBN:9781450316514

DOI:10.1145/2382196

General Chair:
Ting Yu
North Carolina State University, USA
,
Program Chairs:
George Danezis
Microsoft Research Cambridge, UK
,
Virgil Gligor
Carnegie Mellon University, USA

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSAC: ACM Special Interest Group on Security, Audit, and Control

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 October 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CCS'12

Sponsor:

SIGSAC

CCS'12: the ACM Conference on Computer and Communications Security

October 16 - 18, 2012

North Carolina, Raleigh, USA

Acceptance Rates

Overall Acceptance Rate 1,261 of 6,999 submissions, 18%

Upcoming Conference

CCS '25

Sponsor:
sigsac

ACM SIGSAC Conference on Computer and Communications Security

October 13 - 17, 2025

Taipei , Taiwan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

109
Total Citations
View Citations
1,347
Total Downloads

Downloads (Last 12 months)111
Downloads (Last 6 weeks)12

Reflects downloads up to 28 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Goualard F(2024)MicroFloatingPoints.jl: providing very small IEEE 754-compliant floating-point typesJournal of Open Source Software10.21105/joss.070509:101(7050)Online publication date: Oct-2024
https://doi.org/10.21105/joss.07050
Cho HFroelicher DDokmai NNandi ASadhuka SHong MBerger B(2024)Privacy-Enhancing Technologies in Biomedical Data ScienceAnnual Review of Biomedical Data Science10.1146/annurev-biodatasci-120423-1201077:1(317-343)Online publication date: 23-Aug-2024
https://doi.org/10.1146/annurev-biodatasci-120423-120107
Song PSarathy JShoemate MVadhan S(2024)"I inherently just trust that it works": Investigating Mental Models of Open-Source Libraries for Differential PrivacyProceedings of the ACM on Human-Computer Interaction10.1145/36870118:CSCW2(1-39)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3687011
Ratliff ZVadhan SLuo BLiao XXu JKirda ELie D(2024)A Framework for Differential Privacy Against Timing AttacksProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security10.1145/3658644.3690206(3615-3629)Online publication date: 2-Dec-2024
https://dl.acm.org/doi/10.1145/3658644.3690206
Jin JChuengsatiansup CMurray TRubinstein BYarom YOhrimenko OLuo BLiao XXu JKirda ELie D(2024)Elephants Do Not Forget: Differential Privacy with State Continuity for Privacy BudgetProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security10.1145/3658644.3670281(1909-1923)Online publication date: 2-Dec-2024
https://dl.acm.org/doi/10.1145/3658644.3670281
Zhang YLiang XDu RTian J(2024)DP-Discriminator: A Differential Privacy Evaluation Tool Based on GANProceedings of the 21st ACM International Conference on Computing Frontiers10.1145/3649153.3649211(285-293)Online publication date: 7-May-2024
https://dl.acm.org/doi/10.1145/3649153.3649211
Liu JGong NDas H(2024)Two Birds With One Stone: Differential Privacy by Low-Power SRAM MemoryIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2024.338263021:6(5706-5719)Online publication date: Dec-2024
https://doi.org/10.1109/TDSC.2024.3382630
Li XLiu WFeng HHuang KHu YLiu JRen KQin Z(2024)Privacy Enhancement Via Dummy Points in the Shuffle ModelIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2023.326316221:3(1001-1016)Online publication date: May-2024
https://doi.org/10.1109/TDSC.2023.3263162
Küchler NOpel ELycklama HViand AHithnawi A(2024)Cohere: Managing Differential Privacy in Large Scale Systems2024 IEEE Symposium on Security and Privacy (SP)10.1109/SP54263.2024.00122(991-1008)Online publication date: 19-May-2024
https://doi.org/10.1109/SP54263.2024.00122
Hussain SMcDaniel PGandhi AGhose KGopalan KLee DLiu YLiu ZMu SZadok E(2024)Verifiable Sustainability in Data CentersIEEE Security & Privacy10.1109/MSEC.2024.337248822:6(62-74)Online publication date: Dec-2024
https://doi.org/10.1109/MSEC.2024.3372488
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents