research-article

LightDP: towards automating differential privacy proofs

Authors:

Daniel KiferAuthors Info & Claims

POPL '17: Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages

Pages 888 - 901

https://doi.org/10.1145/3009837.3009884

Published: 01 January 2017 Publication History

Abstract

The growing popularity and adoption of differential privacy in academic and industrial settings has resulted in the development of increasingly sophisticated algorithms for releasing information while preserving privacy. Accompanying this phenomenon is the natural rise in the development and publication of incorrect algorithms, thus demonstrating the necessity of formal verification tools. However, existing formal methods for differential privacy face a dilemma: methods based on customized logics can verify sophisticated algorithms but come with a steep learning curve and significant annotation burden on the programmers, while existing programming platforms lack expressive power for some sophisticated algorithms.

In this paper, we present LightDP, a simple imperative language that strikes a better balance between expressive power and usability. The core of LightDP is a novel relational type system that separates relational reasoning from privacy budget calculations. With dependent types, the type system is powerful enough to verify sophisticated algorithms where the composition theorem falls short. In addition, the inference engine of LightDP infers most of the proof details, and even searches for the proof with minimal privacy cost when multiple proofs exist. We show that LightDP verifies sophisticated algorithms with little manual effort.

References

[1]

A. Aiken and E. L. Wimmers. Type inclusion constraints and type inference. In FPLCA, pages 31–41, 1993.

Digital Library

[2]

G. Barthe and F. Olmedo. Beyond differential privacy: Composition theorems and relational logic for f-divergences between probabilistic programs. In ICALP, pages 49–60, 2013.

Digital Library

[3]

G. Barthe, B. Köpf, F. Olmedo, and S. Zanella Béguelin. Probabilistic relational reasoning for differential privacy. In Proceedings of the 39th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pages 97–110, 2012.

Digital Library

[4]

G. Barthe, G. Danezis, B. Grégoire, C. Kunz, and S. Zanella-Béguelin. Verified computational differential privacy with applications to smart metering. In 2013 IEEE 26th Computer Security Foundations Symposium, pages 287–301, 2013.

Digital Library

[5]

G. Barthe, M. Gaboardi, E. J. G. Arias, J. Hsu, C. Kunz, and P. Y. Strub. Proving differential privacy in hoare logic. In 2014 IEEE 27th Computer Security Foundations Symposium, pages 411–424, 2014.

Digital Library

[6]

G. Barthe, M. Gaboardi, E. J. G. Arias, J. Hsu, A. Roth, and P. Strub. Higher-order approximate relational refinement types for mechanism design and differential privacy. In POPL, 2015.

Digital Library

[7]

N. Bjørner, A.-D. Phan, and L. Fleckenstein. νZ — An Optimizing SMT Solver, pages 194–199. 2015.

[8]

H. Chan, E. Shi, and D. Song. Private and continual release of statistics. ACM Transactions on Information and System Security, 14(3), 2011.

Digital Library

[9]

Y. Chen and A. Machanavajjhala. On the privacy properties of variants on the sparse vector technique. http://arxiv.org/abs/1508.07306, 2015.

[10]

L. D’Antoni, M. Gaboardi, E. J. Gallego Arias, A. Haeberlen, and B. Pierce. Sensitivity analysis using type-based constraints. In Proceedings of the 1st Annual Workshop on Functional Programming Concepts in Domainspecific Languages, pages 43–50, 2013.

Digital Library

[11]

L. M. de Moura and N. Bjørner. Z3: An efficient SMT solver. In Conf. on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), 2008.

Digital Library

[12]

C. Dwork and A. Roth. The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science, 9(3–4):211– 407, 2014. ISSN 1551-305X.

Digital Library

[13]

C. Dwork, K. Kenthapadi, F. McSherry, I. Mironov, and M. Naor. Our data, ourselves: Privacy via distributed noise generation. In EUROCRYPT, pages 486–503, 2006a. C. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating noise to sensitivity in private data analysis. In TCC, 2006b. H. Ebadi, D. Sands, and G. Schneider. Differential privacy: Now it’s getting personal. In POPL, 2015.

Digital Library

[14]

U. Erlingsson, V. Pihur, and A. Korolova. Rappor: Randomized aggregatable privacy-preserving ordinal response. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security, CCS ’14, 2014.

Digital Library

[15]

M. Gaboardi, A. Haeberlen, J. Hsu, A. Narayan, and B. C. Pierce. Linear dependent types for differential privacy. In Proceedings of the 40th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’13, pages 357–370, 2013.

Digital Library

[16]

A. Greenberg. Apple’s ‘differential privacy’ is about collecting your data – but not Your data. Wired, https://www.wired.com/2016/ 06/apples-differential-privacy-collecting-data/, 2016.

[17]

C. Haack and J. B. Wells. Type error slicing in implicitly typed higher-order languages. Science of Computer Programming, 50(1–3):189–224, 2004.

Digital Library

[18]

D. Kifer and A. Machanavajjhala. Pufferfish: A framework for mathematical privacy definitions. ACM Trans. Database Syst., 39(1):3:1–3:36, 2014.

Digital Library

[19]

D. Kozen. Semantics of probabilistic programs. Journal of Computer and System Sciences, 22(3):328 – 350, 1981.

[20]

K. R. M. Leino. Dafny: An automatic program verifier for functional correctness. In Proceedings of the 16th International Conference on Logic for Programming, Artificial Intelligence, and Reasoning, pages 348–370, 2010.

Digital Library

[21]

M. Lyu, D. Su, and N. Li. Understanding the sparse vector technique for differential privacy. https://arxiv.org/abs/1603.01699, 2016.

[22]

A. Machanavajjhala, D. Kifer, J. Abowd, J. Gehrke, and L. Vilhuber. Privacy: From theory to practice on the map. In Proceedings of the IEEE International Conference on Data Engineering (ICDE), pages 277–286, 2008.

Digital Library

[23]

P. Martin-Löf. Intuitionistic type theory. Naples: Bibliopolis, 76, 1984.

[24]

F. McSherry and K. Talwar. Mechanism design via differential privacy. In Proceedings of the 48th Annual IEEE Symposium on Foundations of Computer Science, pages 94–103, 2007.

Digital Library

[25]

F. D. McSherry. Privacy integrated queries: An extensible platform for privacy-preserving data analysis. In Proceedings of the 2009 ACM SIGMOD International Conference on Management of Data, pages 19– 30, 2009.

Digital Library

[26]

P. Mohan, A. Thakurta, E. Shi, D. Song, and D. Culler. Gupt: Privacy preserving data analysis made easy. In Proceedings of the ACM SIGMOD International Conference on Management of Data, 2012.

Digital Library

[27]

J. Reed and B. C. Pierce. Distance makes the types grow stronger: A calculus for differential privacy. In Proceedings of the 15th ACM SIGPLAN International Conference on Functional Programming, ICFP ’10, pages 157–168, 2010.

Digital Library

[28]

A. Roth. The sparse vector technique. http://www.cis.upenn. edu/˜aaroth/courses/slides/Lecture11.pdf, 2011.

[29]

I. Roy, S. Setty, A. Kilzer, V. Shmatikov, and E. Witchel. Airavat: Security and privacy for MapReduce. In NSDI, 2010.

Digital Library

[30]

M. C. Tschantz, D. Kaynar, and A. Datta. Formal verification of differential privacy for interactive systems (extended abstract). Electron. Notes Theor. Comput. Sci., 276:61–79, Sept. 2011.

Digital Library

[31]

M. Wand. A simple algorithm and proof for type inference. Fundamenta Informaticae, 10:115–122, 1987.

[32]

L. Xu, K. Chatzikokolakis, and H. Lin. Metrics for Differential Privacy in Concurrent Systems, pages 199–215. 2014.

[33]

D. Zhang and D. Kifer. LightDP: Towards automating differential privacy proofs. CoRR, abs/1607.08228, 2016.

Digital Library

[34]

J. Zhang, X. Xiao, and X. Xie. Privtree: A differentially private algorithm for hierarchical decompositions. In SIGMOD, 2016.

Digital Library

Cited By

Bao JD'Osualdo EFarzan A(2025)Bluebell: An Alliance of Relational Lifting and Independence for Probabilistic ReasoningProceedings of the ACM on Programming Languages10.1145/37048949:POPL(1719-1749)Online publication date: 9-Jan-2025
https://dl.acm.org/doi/10.1145/3704894
Lobo-Vesga ERusso AGaboardi MCortiñas C(2024)Sensitivity by ParametricityProceedings of the ACM on Programming Languages10.1145/36897268:OOPSLA2(415-441)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3689726
Zhang YLiang XDu RTian J(2024)DP-Discriminator: A Differential Privacy Evaluation Tool Based on GANProceedings of the 21st ACM International Conference on Computing Frontiers10.1145/3649153.3649211(285-293)Online publication date: 7-May-2024
https://dl.acm.org/doi/10.1145/3649153.3649211
Show More Cited By

Recommendations

LightDP: towards automating differential privacy proofs
POPL '17

The growing popularity and adoption of differential privacy in academic and industrial settings has resulted in the development of increasingly sophisticated algorithms for releasing information while preserving privacy. Accompanying this phenomenon is ...
Linear dependent types for differential privacy
POPL '13: Proceedings of the 40th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages

Differential privacy offers a way to answer queries about sensitive information while providing strong, provable privacy guarantees, ensuring that the presence or absence of a single individual in the database has a negligible statistical effect on the ...
Linear dependent types for differential privacy
POPL '13

Differential privacy offers a way to answer queries about sensitive information while providing strong, provable privacy guarantees, ensuring that the presence or absence of a single individual in the database has a negligible statistical effect on the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

POPL '17: Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages

January 2017

901 pages

ISBN:9781450346603

DOI:10.1145/3009837

General Chair:
Giuseppe Castagna
Paris Diderot University, France / CNRS, France
,
Program Chair:
Andrew D. Gordon
Microsoft Research, UK / University of Edinburgh, UK

ACM SIGPLAN Notices Volume 52, Issue 1
POPL '17
January 2017
901 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/3093333
Editor:
Matthew Fluet
Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGPLAN: ACM Special Interest Group on Programming Languages

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 January 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

POPL '17

Sponsor:

SIGPLAN

POPL '17: The 44th Annual ACM SIGPLAN Symposium on Principles of Programming Languages

January 15 - 21, 2017

Paris, France

Acceptance Rates

Overall Acceptance Rate 824 of 4,130 submissions, 20%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

55
Total Citations
View Citations
533
Total Downloads

Downloads (Last 12 months)41
Downloads (Last 6 weeks)0

Reflects downloads up to 20 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bao JD'Osualdo EFarzan A(2025)Bluebell: An Alliance of Relational Lifting and Independence for Probabilistic ReasoningProceedings of the ACM on Programming Languages10.1145/37048949:POPL(1719-1749)Online publication date: 9-Jan-2025
https://dl.acm.org/doi/10.1145/3704894
Lobo-Vesga ERusso AGaboardi MCortiñas C(2024)Sensitivity by ParametricityProceedings of the ACM on Programming Languages10.1145/36897268:OOPSLA2(415-441)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3689726
Zhang YLiang XDu RTian J(2024)DP-Discriminator: A Differential Privacy Evaluation Tool Based on GANProceedings of the 21st ACM International Conference on Computing Frontiers10.1145/3649153.3649211(285-293)Online publication date: 7-May-2024
https://dl.acm.org/doi/10.1145/3649153.3649211
Lu YMagdon-Ismail MWei YZikas V(2024)Eureka: A General Framework for Black-box Differential Privacy Estimators2024 IEEE Symposium on Security and Privacy (SP)10.1109/SP54263.2024.00166(913-931)Online publication date: 19-May-2024
https://doi.org/10.1109/SP54263.2024.00166
Oakley LHoltzen SOprea A(2024)Synthesizing Tight Privacy and Accuracy Bounds via Weighted Model Counting2024 IEEE 37th Computer Security Foundations Symposium (CSF)10.1109/CSF61375.2024.00048(449-463)Online publication date: 8-Jul-2024
https://doi.org/10.1109/CSF61375.2024.00048
Künnemann RPatrignani MCecchetti E(2024)Computationally Bounded Robust Compilation and Universally Composable Security2024 IEEE 37th Computer Security Foundations Symposium (CSF)10.1109/CSF61375.2024.00024(265-278)Online publication date: 8-Jul-2024
https://doi.org/10.1109/CSF61375.2024.00024
Bell ZGoldwasser SKim MWatson J(2024)Certifying Private Probabilistic MechanismsAdvances in Cryptology – CRYPTO 202410.1007/978-3-031-68391-6_11(348-386)Online publication date: 18-Aug-2024
https://dl.acm.org/doi/10.1007/978-3-031-68391-6_11
Toro MDarais DAbuah CNear JÁrquez DOlmedo FTanter É(2023)Contextual Linear Types for Differential PrivacyACM Transactions on Programming Languages and Systems10.1145/358920745:2(1-69)Online publication date: 17-May-2023
https://dl.acm.org/doi/10.1145/3589207
Chadha RSistla AViswanathan MBhusal BMeng WJensen CCremers CKirda E(2023)Deciding Differential Privacy of Online Algorithms with Multiple VariablesProceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security10.1145/3576915.3623170(1761-1775)Online publication date: 15-Nov-2023
https://dl.acm.org/doi/10.1145/3576915.3623170
Lokna JParadis ADimitrov DVechev MMeng WJensen CCremers CKirda E(2023)Group and Attack: Auditing Differential PrivacyProceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security10.1145/3576915.3616607(1905-1918)Online publication date: 15-Nov-2023
https://dl.acm.org/doi/10.1145/3576915.3616607
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents