Cited By
View all- Yang MGuo TZhu TTjuawinata IZhao JLam K(2024)Local differential privacy and its applicationsComputer Standards & Interfaces10.1016/j.csi.2023.10382789:COnline publication date: 25-Jun-2024
Compressive Sensing Approaches for Sparse Distribution Estimation Under Local Privacy
Authors: 
Zhongzheng Xiong, Jialin Sun, Xiaojun Mao, Jian Wang, Ying Shan, Zengfeng HuangAuthors Info & Claims
Zhongzheng Xiong, Jialin Sun, Xiaojun Mao, Jian Wang, Ying Shan, Zengfeng HuangAuthors Info & ClaimsPages 599 - 609
Abstract
Recent years, local differential privacy (LDP) has been adopted by many web service providers like Google [23], Apple [33] and Microsoft [15] to collect and analyse users’ data privately. In this paper, we consider the problem of discrete distribution estimation under local differential privacy constraints. Distribution estimation is one of the most fundamental estimation problems, which is widely studied in both non-private and private settings. In the local model, private mechanisms with provably optimal sample complexity are known. However, they are optimal only in the worst-case sense; their sample complexity is proportional to the size of the entire universe, which could be huge in practice. In this paper, we consider sparse or approximately sparse (e.g. highly skewed) distribution, and show that the number of samples needed could be significantly reduced. This problem has been studied recently [1], but they only consider strict sparse distributions and the high privacy regime. We propose new privatization mechanisms based on compressive sensing. Our methods work for approximately sparse distributions and medium privacy, and have optimal sample and communication complexity.
References
[1]
Jayadev Acharya, Peter Kairouz, Yuhan Liu, and Ziteng Sun. 2021. Estimating Sparse Discrete Distributions Under Privacy and Communication Constraints. In Algorithmic Learning Theory. PMLR, 79–98.
[2]
Jayadev Acharya and Ziteng Sun. 2019. Communication Complexity in Locally Private Distribution Estimation and Heavy Hitters. In International Conference on Machine Learning. 51–60.
[3]
Jayadev Acharya, Ziteng Sun, and Huanyu Zhang. 2018. Hadamard response: Estimating distributions privately, efficiently, and with little communication. arXiv preprint arXiv:1802.04705(2018).
[4]
Richard Baraniuk, Mark Davenport, Ronald DeVore, and Michael Wakin. 2008. A simple proof of the restricted isometry property for random matrices. Constructive Approximation 28, 3 (2008), 253–263.
[5]
Leighton Pate Barnes, Wei-Ning Chen, and Ayfer Özgür. 2020. Fisher information under local differential privacy. IEEE Journal on Selected Areas in Information Theory (2020).
[6]
Raef Bassily. 2019. Linear Queries Estimation with Local Differential Privacy. In The 22nd International Conference on Artificial Intelligence and Statistics. 721–729.
[7]
Raef Bassily, Kobbi Nissim, Uri Stemmer, and Abhradeep Guha Thakurta. 2017. Practical locally private heavy hitters. In Advances in Neural Information Processing Systems. 2288–2296.
[8]
Amos Beimel, Kobbi Nissim, and Eran Omri. 2008. Distributed private data analysis: Simultaneously solving how and what. In Annual International Cryptology Conference. Springer, 451–468.
[9]
Mark Bun, Jelani Nelson, and Uri Stemmer. 2018. Heavy hitters and the structure of local privacy. In Proceedings of the 37th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems. 435–447.
[10]
T Tony Cai and Anru Zhang. 2013. Sparse representation of a polytope and recovery of sparse signals and low-rank matrices. IEEE transactions on information theory 60, 1 (2013), 122–132.
[11]
Emmanuel J Candès, Justin Romberg, and Terence Tao. 2006. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on information theory 52, 2 (2006), 489–509.
[12]
Emmanuel J Candes, Justin K Romberg, and Terence Tao. 2006. Stable signal recovery from incomplete and inaccurate measurements. Communications on Pure and Applied Mathematics: A Journal Issued by the Courant Institute of Mathematical Sciences 59, 8 (2006), 1207–1223.
[13]
Emmanuel J Candes and Terence Tao. 2005. Decoding by linear programming. IEEE transactions on information theory 51, 12 (2005), 4203–4215.
[14]
Wei-Ning Chen, Peter Kairouz, and Ayfer Özgür. 2020. Breaking the Communication-Privacy-Accuracy Trilemma. arXiv preprint arXiv:2007.11707(2020).
[15]
Bolin Ding, Janardhan Kulkarni, and Sergey Yekhanin. 2017. Collecting Telemetry Data Privately. In Advances in Neural Information Processing Systems. Curran Associates, Inc.
[16]
David L Donoho. 2006. Compressed sensing. IEEE Transactions on information theory 52, 4 (2006), 1289–1306.
[17]
Marco F Duarte and Richard G Baraniuk. 2011. Kronecker compressive sensing. IEEE Transactions on Image Processing 21, 2 (2011), 494–504.
[18]
Devdatt P Dubhashi and Desh Ranjan. 1996. Balls and bins: A study in negative dependence. BRICS Report Series 3, 25 (1996).
[19]
John Duchi and Ryan Rogers. 2019. Lower bounds for locally private estimation via communication complexity. In Conference on Learning Theory. PMLR, 1161–1191.
[20]
John C Duchi, Michael I Jordan, and Martin J Wainwright. 2013. Local privacy and statistical minimax rates. In 2013 IEEE 54th Annual Symposium on Foundations of Computer Science. IEEE, 429–438.
[21]
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. 2006. Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference. Springer, 265–284.
[22]
Cynthia Dwork, Aaron Roth, 2014. The algorithmic foundations of differential privacy.Foundations and Trends in Theoretical Computer Science 9, 3-4(2014), 211–407.
[23]
Úlfar Erlingsson, Vasyl Pihur, and Aleksandra Korolova. 2014. Rappor: Randomized aggregatable privacy-preserving ordinal response. In Proceedings of the 2014 ACM SIGSAC conference on computer and communications security. 1054–1067.
[24]
Vitaly Feldman and Kunal Talwar. 2021. Lossless Compression of Efficient Private Local Randomizers. arXiv preprint arXiv:2102.12099(2021).
[25]
Peter Kairouz, Keith Bonawitz, and Daniel Ramage. 2016. Discrete distribution estimation under local privacy. arXiv preprint arXiv:1602.07387(2016).
[26]
Peter Kairouz, Sewoong Oh, and Pramod Viswanath. 2014. Extremal mechanisms for local differential privacy. In Advances in neural information processing systems. 2879–2887.
[27]
Sudeep Kamath, Alon Orlitsky, Dheeraj Pichapati, and Ananda Theertha Suresh. 2015. On learning distributions from their samples. In Conference on Learning Theory. 1066–1100.
[28]
Shiva Prasad Kasiviswanathan, Homin K Lee, Kobbi Nissim, Sofya Raskhodnikova, and Adam Smith. 2011. What can we learn privately?SIAM J. Comput. 40, 3 (2011), 793–826.
[29]
Erich L Lehmann and George Casella. 2006. Theory of point estimation. Springer Science & Business Media.
[30]
Adriano Pastore and Michael Gastpar. 2016. Locally differentially-private distribution estimation. In 2016 IEEE International Symposium on Information Theory (ISIT). Ieee, 2694–2698.
[31]
Ingo Roth, Axel Flinth, Richard Kueng, Jens Eisert, and Gerhard Wunder. 2018. Hierarchical restricted isometry property for Kronecker product measurements. In 2018 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton). IEEE, 632–638.
[32]
Ingo Roth, Martin Kliesch, Gerhard Wunder, and Jens Eisert. 2016. Reliable recovery of hierarchically sparse signals and application in machine-type communications. arXiv preprint arXiv:1612.07806(2016).
[33]
Apple Differential Privacy Team. 2017. Learning with privacy at scale. (2017).
[34]
Joel A Tropp. 2004. Greed is good: Algorithmic results for sparse approximation. IEEE Transactions on Information theory 50, 10 (2004), 2231–2242.
[35]
David Wajc. 2017. Negative association: definition, properties, and applications. Manuscript, available from https://goo. gl/j2ekqM (2017).
[36]
Di Wang and Jinhui Xu. 2019. On sparse linear regression in the local differential privacy model. In International Conference on Machine Learning. PMLR, 6628–6637.
[37]
Shaowei Wang, Liusheng Huang, Pengzhan Wang, Yiwen Nie, Hongli Xu, Wei Yang, Xiang-Yang Li, and Chunming Qiao. 2016. Mutual information optimally local private discrete distribution estimation. arXiv preprint arXiv:1607.08025(2016).
[38]
Tianhao Wang, Jeremiah Blocki, Ninghui Li, and Somesh Jha. 2017. Locally Differentially Private Protocols for Frequency Estimation. In 26th USENIX Security Symposium (USENIX Security 17). USENIX Association.
[39]
Tianhao Wang, Ninghui Li, and Somesh Jha. 2019. Locally differentially private heavy hitter identification. IEEE Transactions on Dependable and Secure Computing (2019).
[40]
Stanley L Warner. 1965. Randomized response: A survey technique for eliminating evasive answer bias. J. Amer. Statist. Assoc. 60, 309 (1965), 63–69.
[41]
Min Ye and Alexander Barg. 2018. Optimal schemes for discrete distribution estimation under locally differential privacy. IEEE Transactions on Information Theory 64, 8 (2018), 5662–5676.
Index Terms
- Compressive Sensing Approaches for Sparse Distribution Estimation Under Local Privacy
Index terms have been assigned to the content through auto-classification.
Recommendations
Image compressive sensing via Truncated Schatten-p Norm regularization
Low-rank property as a useful image prior has attracted much attention in image processing communities. Recently, a nonlocal low-rank regularization (NLR) approach toward exploiting low-rank property has shown the state-of-the-art performance in ...
Privacy-aware data publishing against sparse estimation attack
Recently a number of privacy-preserving data publishing techniques have been proposed to protect the privacy of released data. In this paper, we study how to protect unreleased data privacy during data publishing. Our experiment results show that the ...
Utility-optimized local differential privacy mechanisms for distribution estimation
SEC'19: Proceedings of the 28th USENIX Conference on Security SymposiumLDP (Local Differential Privacy) has been widely studied to estimate statistics of personal data (e.g., distribution underlying the data) while protecting users' privacy. Although LDP does not require a trusted third party, it regards all personal data ...
Comments
Information & Contributors
Information
Published In
April 2022
3764 pages
ISBN:9781450390965
DOI:10.1145/3485447
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 25 April 2022
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- Science and Technology Commission of Shanghai Municipality Project Grant
- National Natural Science Foundation of China Grant
- Shanghai Science and Technology Commission Grant
Conference
WWW '22
Sponsor:
Acceptance Rates
Overall Acceptance Rate 1,899 of 8,196 submissions, 23%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 269Total Downloads
- Downloads (Last 12 months)49
- Downloads (Last 6 weeks)7
Reflects downloads up to 27 Jul 2024
Other Metrics
Citations
Cited By
View all- Yang MGuo TZhu TTjuawinata IZhao JLam K(2024)Local differential privacy and its applicationsComputer Standards & Interfaces10.1016/j.csi.2023.10382789:COnline publication date: 25-Jun-2024
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format