research-article

Verifiable differential privacy

Authors:

Antonis Papadimitriou,

Andreas HaeberlenAuthors Info & Claims

EuroSys '15: Proceedings of the Tenth European Conference on Computer Systems

Article No.: 28, Pages 1 - 14

https://doi.org/10.1145/2741948.2741978

Published: 17 April 2015 Publication History

Abstract

Working with sensitive data is often a balancing act between privacy and integrity concerns. Consider, for instance, a medical researcher who has analyzed a patient database to judge the effectiveness of a new treatment and would now like to publish her findings. On the one hand, the patients may be concerned that the researcher's results contain too much information and accidentally leak some private fact about themselves; on the other hand, the readers of the published study may be concerned that the results contain too little information, limiting their ability to detect errors in the calculations or flaws in the methodology.

This paper presents VerDP, a system for private data analysis that provides both strong integrity and strong differential privacy guarantees. VerDP accepts queries that are written in a special query language, and it processes them only if a) it can certify them as differentially private, and if b) it can prove the integrity of the result in zero knowledge. Our experimental evaluation shows that VerDP can successfully process several different queries from the differential privacy literature, and that the cost of generating and verifying the proofs is practical: for example, a histogram query over a 63,488-entry data set resulted in a 20 kB proof that took 32 EC2 instances less than two hours to generate, and that could be verified on a single machine in about one second.

Supplementary Material

MP4 File (a28-sidebyside.mp4)

Download
864.89 MB

References

[1]

Barbaro, M., Zeller, T., and Hansell, S. A face is exposed for AOL searcher No. 4417749. The New York Times (August 9, 2006). http://www.nytimes.com/2006/08/09/technology/09aol.html.

[2]

Barthe, G., Köpf, B., Olmedo, F., and Zanella Béguelin, S. Probabilistic relational reasoning for differential privacy. In Proc. POPL (2012).

Digital Library

[3]

Bell, R. M., and Koren, Y. Lessons from the Netflix prize challenge. SIGKDD Explor. Newsl. 9, 2 (2007).

Digital Library

[4]

Ben-Sasson, E., Chiesa, A., Genkin, D., Tromer, E., and Virza, M. SNARKs for C: Verifying program executions succinctly and in zero knowledge. In Proc. CRYPTO (2013).

[5]

Ben-Sasson, E., Chiesa, A., Tromer, E., and Virza, M. Succinct non-interactive zero knowledge for a von neumann architecture. In Proc. USENIX Security (2014).

Digital Library

[6]

Blake, H., Watt, H., and Winnett, R. Millions of surgery patients at risk in drug research fraud scandal. The Telegraph (March 3, 2011). http://www.telegraph.co.uk/health/8360667/Millions-of-surgery-patients-at-risk-in-drug-research-fraud-scandal.html.

[7]

Blum, A., Dwork, C., McSherry, F., and Nissim, K. Practical privacy: the SuLQ framework. In Proc. PODS (2005).

Digital Library

[8]

Braun, B., Feldman, A. J., Ren, Z., Setty, S., Blumberg, A. J., and Walfish, M. Verifying computations with state. In Proc. SOSP (2013).

Digital Library

[9]

Chawla, S., Dwork, C., McSherry, F., Smith, A., and Wee, H. Toward privacy in public databases. In Proc. TCC (2005).

Digital Library

[10]

Chen, R., Reznichenko, A., Francis, P., and Gehrke, J. Towards statistical queries over distributed private user data. In Proc. NSDI (2012).

Digital Library

[11]

Cormode, G., Mitzenmacher, M., and Thaler, J. Practical verified computation with streaming interactive proofs. In Proc. ITCS (2012).

Digital Library

[12]

Deer, B. MMR doctor Andrew Wakefield fixed data on autism. The Sunday Times (February 8, 2009). http://www.thesundaytimes.co.uk/sto/public/news/article148992.ece.

[13]

Dwork, C. Differential privacy: A survey of results. In Proc. TAMC (2008).

Digital Library

[14]

Dwork, C., Kenthapadi, K., McSherry, F., Mironov, I., and Naor, M. Our data, ourselves: Privacy via distributed noise generation. In Proc. EUROCRYPT (2006).

Digital Library

[15]

Dwork, C., McSherry, F., Nissim, K., and Smith, A. Calibrating noise to sensitivity in private data analysis. In Proc. TCC (2006).

Digital Library

[16]

Fournet, C., Kohlweiss, M., Danezis, G., and Luo, Z. ZQL: A compiler for privacy-preserving data processing. In Proc. USENIX Security (2013).

Digital Library

[17]

Fredrikson, M., and Livshits, B. ZØ: An optimizing distributing zero-knowledge compiler. In Proc. USENIX Security (2014).

Digital Library

[18]

Gaboardi, M., Haeberlen, A., Hsu, J., Narayan, A., and Pierce, B. C. Linear dependent types for differential privacy. In Proc. POPL (2013).

Digital Library

[19]

Gennaro, R., Gentry, C., Parno, B., and Raykova, M. Quadratic span programs and succinct NIZKs without PCPs. In Proc. EUROCRYPT (2013).

[20]

Goldwasser, S., Kalai, Y. T., and Rothblum, G. N. Delegating computation: Interactive proofs for muggles. In Proc. STOC (2008).

Digital Library

[21]

Haeberlen, A., Pierce, B. C., and Narayan, A. Differential privacy under fire. In Proc. USENIX Security (2011).

Digital Library

[22]

Hawblitzel, C., Howell, J., Lorch, J. R., Narayan, A., Parno, B., Zhang, D., and Zill, B. Ironclad apps: End-to-end security via automated full-system verification. In Proc. OSDI (2014).

Digital Library

[23]

Herndon, T., Ash, M., and Pollin, R. Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff. Working paper 322, Political Economy Research Institute, University of Massachusetts Amherst, 2013. http://www.peri.umass.edu/fileadmin/pdf/working_papers/working_papers_301-350/WP322.pdf.

[24]

Hsu, J., Gaboardi, M., Haeberlen, A., Khanna, S., Narayan, A., Pierce, B. C., and Roth, A. Differential privacy: An economic method for choosing epsilon. In Proc. CSF (2014).

Digital Library

[25]

ICPSR Data Deposit at the University of Michigan. http://www.icpsr.umich.edu/icpsrweb/deposit/.

[26]

Integrating Data for Analysis, Anonymization and SHaring. http://idash.ucsd.edu/.

[27]

Interlandi, J. An unwelcome discovery. The New York Times (October 22, 2006). www.nytimes.com/2006/10/22/magazine/22sciencefraud.html.

[28]

Integrated Public Use Microdata Series at the Minnesota Population Center. https://www.ipums.org/.

[29]

Ishai, Y., Kushilevitz, E., and Ostrovsky, R. Efficient arguments without short PCPs. In Proc. CCC (2007).

Digital Library

[30]

McSherry, F. Privacy Integrated Queries. In Proc. SIGMOD (2009).

[31]

Meiklejohn, S., Erway, C. C., Küpçü, A., Hinkle, T., and Lysyanskaya, A. ZKPDL: A language-based system for efficient zero-knowledge proofs and electronic cash. In Proc. USENIX Security (2010).

Digital Library

[32]

Mironov, I. On significance of the least significant bits for differential privacy. In Proc. CCS (2012).

Digital Library

[33]

Mironov, I., Pandey, O., Reingold, O., and Vadhan, S. Computational differential privacy. In Proc. CRYPTO (2009).

Digital Library

[34]

Mohan, P., Thakurta, A., Shi, E., Song, D., and Culler, D. GUPT: Privacy preserving data analysis made easy. In Proc. SIGMOD (2012).

Digital Library

[35]

Narayan, A., Feldman, A., Papadimitriou, A., and Haeberlen, A. Verifiable differential privacy. Tech. Rep. MS-CIS-15-05, Department of Computer and Information Science, University of Pennsylvania, Mar. 2015.

[36]

Narayan, A., and Haeberlen, A. DJoin: Differentially private join queries over distributed databases. In Proc. OSDI (2012).

Digital Library

[37]

Narayanan, A., and Shmatikov, V. Robust de-anonymization of large sparse datasets. In Proc. S&P (2008).

Digital Library

[38]

Parno, B., Gentry, C., Howell, J., and Raykova, M. Pinocchio: Nearly practical verifiable computation. In Proc. S&P (2013).

Digital Library

[39]

Reed, J., and Pierce, B. C. Distance makes the types grow stronger: A calculus for differential privacy. In Proc. ICFP (2010).

Digital Library

[40]

Roy, I., Setty, S., Kilzer, A., Shmatikov, V., and Witchel, E. Airavat: Security and privacy for MapReduce. In Proc. NSDI (2010).

Digital Library

[41]

Setty, S., Braun, B., Vu, V., Blumberg, A. J., Parno, B., and Walfish, M. Resolving the conflict between generality and plausibility in verified computation. In Proc. EuroSys (2013).

Digital Library

[42]

Setty, S., McPherson, R., Blumberg, A. J., and Walfish, M. Making argument systems for outsourced computation practical (sometimes). In Proc. NDSS (2012).

[43]

Setty, S., Vu, V., Panpalia, N., Braun, B., Blumberg, A. J., and Walfish, M. Taking proof-based verified computation a few steps closer to practicality. In Proc. USENIX Security (2012).

Digital Library

[44]

Sweeney, L. k-anonymity: A model for protecting privacy. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 10, 05 (2002).

Digital Library

[45]

Thaler, J. Time-optimal interactive proofs for circuit evaluation. In Proc. CRYPTO (2013).

[46]

Thaler, J., Roberts, M., Mitzenmacher, M., and Pfister, H. Verifiable computation with massively parallel interactive proofs. In Proc. HotCloud (2012).

Digital Library

[47]

Vu, V., Setty, S., Blumberg, A. J., and Walfish, M. A hybrid architecture for interactive verifiable computation. In Proc. S&P (2013).

Digital Library

[48]

Wahby, R. S., Setty, S., Ren, Z., Blumberg, A. J., and Walfish, M. Efficient RAM and control flow in verifiable outsourced computation. Cryptology ePrint 2014/674, 2014.

Cited By

Sato TMinamide YStark KTimany ABlazy STabareau N(2025)Formalization of Differential Privacy in Isabelle/HOLProceedings of the 14th ACM SIGPLAN International Conference on Certified Programs and Proofs10.1145/3703595.3705875(67-82)Online publication date: 10-Jan-2025
https://dl.acm.org/doi/10.1145/3703595.3705875
Lauinger JErnstberger JSteinhorst S(2024)zkGen: Policy-to-Circuit Transpiler2024 IEEE International Conference on Blockchain and Cryptocurrency (ICBC)10.1109/ICBC59979.2024.10634440(620-624)Online publication date: 27-May-2024
https://doi.org/10.1109/ICBC59979.2024.10634440
Wei JChen YLuo YLi Z(2024)Range Proof-Based Noise Filtering Mechanism for loT Differential Privacy2024 IEEE Cyber Science and Technology Congress (CyberSciTech)10.1109/CyberSciTech64112.2024.00025(94-99)Online publication date: 5-Nov-2024
https://doi.org/10.1109/CyberSciTech64112.2024.00025
Show More Cited By

Index Terms

Verifiable differential privacy

Recommendations

An Emerging Strategy for Privacy Preserving Databases: Differential Privacy
HCI for Cybersecurity, Privacy and Trust
Abstract
Data De-identification and Differential Privacy are two possible approaches for providing data security and user privacy. Data de-identification is the process where the personal identifiable information of individuals is extracted to create ...
Differential privacy: an exploration of the privacy-utility landscape
Differential Privacy: Now it's Getting Personal
POPL '15: Proceedings of the 42nd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages

Differential privacy provides a way to get useful information about sensitive data without revealing much about any one individual. It enjoys many nice compositionality properties not shared by other approaches to privacy, including, in particular, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

EuroSys '15: Proceedings of the Tenth European Conference on Computer Systems

April 2015

503 pages

ISBN:9781450332385

DOI:10.1145/2741948

General Chair:
Laurent Réveillère
LaBRI, University of Bordeaux, France
,
Program Chairs:
Tim Harris
Oracle Labs, UK
,
Maurice Herlihy
Brown University

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGOPS: ACM Special Interest Group on Operating Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 April 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Conference

EuroSys '15

Sponsor:

SIGOPS

EuroSys '15: Tenth EuroSys Conference 2015

April 21 - 24, 2015

Bordeaux, France

Acceptance Rates

Overall Acceptance Rate 241 of 1,308 submissions, 18%

Upcoming Conference

EuroSys '25

Sponsor:
sigops

Twentieth European Conference on Computer Systems

March 30 - April 3, 2025

Rotterdam , Netherlands

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
748
Total Downloads

Downloads (Last 12 months)49
Downloads (Last 6 weeks)3

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Sato TMinamide YStark KTimany ABlazy STabareau N(2025)Formalization of Differential Privacy in Isabelle/HOLProceedings of the 14th ACM SIGPLAN International Conference on Certified Programs and Proofs10.1145/3703595.3705875(67-82)Online publication date: 10-Jan-2025
https://dl.acm.org/doi/10.1145/3703595.3705875
Lauinger JErnstberger JSteinhorst S(2024)zkGen: Policy-to-Circuit Transpiler2024 IEEE International Conference on Blockchain and Cryptocurrency (ICBC)10.1109/ICBC59979.2024.10634440(620-624)Online publication date: 27-May-2024
https://doi.org/10.1109/ICBC59979.2024.10634440
Wei JChen YLuo YLi Z(2024)Range Proof-Based Noise Filtering Mechanism for loT Differential Privacy2024 IEEE Cyber Science and Technology Congress (CyberSciTech)10.1109/CyberSciTech64112.2024.00025(94-99)Online publication date: 5-Nov-2024
https://doi.org/10.1109/CyberSciTech64112.2024.00025
Bell ZGoldwasser SKim MWatson J(2024)Certifying Private Probabilistic MechanismsAdvances in Cryptology – CRYPTO 202410.1007/978-3-031-68391-6_11(348-386)Online publication date: 18-Aug-2024
https://dl.acm.org/doi/10.1007/978-3-031-68391-6_11
Wang CPujol DNayak KMachanavajjhala ACalandrino JTroncoso C(2023)Private proof-of-stake blockchains using differentially-private stake distortionProceedings of the 32nd USENIX Conference on Security Symposium10.5555/3620237.3620326(1577-1594)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.5555/3620237.3620326
Ernstberger JLauinger JElsheimy FZhou LSteinhorst SCanetti RMiller AGervais ASong D(2023)SoK: Data Sovereignty2023 IEEE 8th European Symposium on Security and Privacy (EuroS&P)10.1109/EuroSP57164.2023.00017(122-143)Online publication date: Jul-2023
https://doi.org/10.1109/EuroSP57164.2023.00017
Munilla Garrido GSedlmeir JBabel M(2022)Towards Verifiable Differentially-Private PollingProceedings of the 17th International Conference on Availability, Reliability and Security10.1145/3538969.3538992(1-11)Online publication date: 23-Aug-2022
https://dl.acm.org/doi/10.1145/3538969.3538992
Impagliazzo RLei RPitassi TSorrell JLeonardi SGupta A(2022)Reproducibility in learningProceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing10.1145/3519935.3519973(818-831)Online publication date: 9-Jun-2022
https://dl.acm.org/doi/10.1145/3519935.3519973
Song BLan QLi YLi G(2022)Distributed Differentially Private Ranking AggregationAdvances in Knowledge Discovery and Data Mining10.1007/978-3-031-05933-9_19(236-248)Online publication date: 10-May-2022
https://doi.org/10.1007/978-3-031-05933-9_19
Kato FCao YYoshikawa M(2021)Preventing Manipulation Attack in Local Differential Privacy Using Verifiable Randomization MechanismData and Applications Security and Privacy XXXV10.1007/978-3-030-81242-3_3(43-60)Online publication date: 14-Jul-2021
https://doi.org/10.1007/978-3-030-81242-3_3
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten